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1.0  Scope 


This  report  provides  the  results  of  the  verification  and  validation  (V&V)  tasks  performed  during 
Phase  1  of  the  Joint  Advanced  Distributed  Simulation  (JADS)  End-To-End  (ETE)  Test. 

1.1  Purpose 

This  report  details  the  results  of  executing  the  V&V  requirements  listed  within  the  ETE  Test 
Activity  Plan,  Appendix  C,  Verification  and  Validation  Plan  for  the  ETE  Test. 

1.2  Verification  and  Validation  Tasks 

The  V&V  tasks  that  were  performed  during  or  prior  to  Phase  1  are  described  in  the  ETE  Test 
Activity  Plan,  Appendix  C,  Verification  and  Validation  Plan  for  the  End-To-End  (ETE)  Test. 

1.3  Verification  and  Validation  Process  Models 


Within  this  report,  reference  is  made  to  steps  enumerated  within  the  Distributed  Interactive 
Simulation  (DIS)  Nine  Step  Process  Model.  This  model  is  shown  below  as  Figure  1. 


Figure  1.  DIS  Nine  Step  VV&A  Process  Model 


The  process  model  and  its  accompanying  Recommended  Practice  for  Distributed  Interactive 
Simulation  —  Verification,  Validation,  and  Accreditation  (Draft-4  November  1996)  form  the 
basis  for  the  verification,  validation  and  accreditation  of  the  ETE  Test  synthetic  environment 
(SE). 
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The  DIS  Nine  Step  Process  Model  was  developed  with  a  conventional,  short-lived,  DIS  exercise 
in  mind,  as  opposed  to  a  test  of  a  major  system,  and  presupposes  a  full  complement  of  funds  and 
personnel  available  at  the  beginning  of  the  exercise  development.  This  disparity  was  brought  to 
the  attention  of  the  developers  of  the  DIS  Nine  Step  Process  Model,  and  the  conclusion  was 
reached  that  since  the  model  was  recommended  and  intended  for  tailoring  to  the  needs  of  the 
user,  the  model  would  continue  to  be  tied  to  the  DIS  exercise  development  and  construction 
process  contained  within  Institute  of  Electrical  and  Electronics  Engineers  (IEEE)  Standard 
1278.3. 

If  one  tailors  the  DIS  Nine  Step  Process  Model  to  the  joint  test  process,  then  the  process  model 
would  appear  as  shown  in  Figure  2. 
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Figure  2.  JADS  ETE  Test  Process  Model 


In  the  JADS  ETE  Test  Process  Model,  test  events,  which  consist  of  the  planning,  construction 
and  assembling  of  the  SE,  integration  and  testing  of  the  SE,  accreditation  of  the  SE,  and  conduct 
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of  the  test  all  proceed  on  the  left  side  from  top  to  bottom.  The  V&V  events,  to  include 
documentation,  proceed  to  the  right  for  each  test  event. 

2.  Applicable  Documents 
2.1  Documents 

ETE  Test  Activity  Plan,  Appendix  C,  Verification  and  Validation  Plan  for  the  End-To-End 
(ETE)  Test 

Department  of  Defense  Verification,  Validation  and  Accreditation  (W&A)  Recommended 
Practices  Guide,  November  1996 

Recommended  Practice  for  Distributed  Interactive  Simulation  —  Verification,  Validation,  and 
Accreditation,  4  November  1996 

Joint  Advanced  Distributed  Simulation  Joint  Test  and  Evaluation  Program  Test  Plan  (PTP), 
February  1996 

3.  Verification  and  Validation  Tools 

The  following  software  was  used  in  the  ETE  Phase  1  V&V: 

JADS  Toolbox 
JADS  Logger 

U.S.  Army  Simulation,  Training  and  Instrumentation  Command  (STRICOM  )  Logger 
POWERSIM™  Version  2.02,  POWERSIM  AS,  Norway 


3 


4.  Verification  Tasks 


This  section  of  the  Phase  1  V  &  V  results  states  the  verification  requirements  and  describes  the 
results  of  performing  the  verification  steps  as  indicated  in  ETE  Test  Activity  Plan,  Appendix  C, 
Verification  and  Validation  Plan  for  the  End-To-End  (ETE)  Test,  Figure  2  JADS  ETE  Test  Nine 
Step  Process  Model. 

4.1  Perform  Compliance  Standards  Verification  (Step  2) 

Initial  compliance  standards  verification  was  performed  during  the  development  of  the  program 
test  plan.  At  this  time  the  models  and  simulations  identified  for  use  in  the  ETE  SE  were  verified 
by  reviewing  simulation  descriptions  as  meeting  DIS  compliance  standards  to  the  extent  required. 

During  Phase  1,  visits  were  made  to  each  site  using  DIS-compliant  simulations  (Janus  and 
Tactical  Army  Fire  Support  Model  [TAFSM])  and  prior  exercises  were  reviewed  to  verify  that 
the  simulations  were  DIS  compliant.  Additionally,  numerous  Janus  simulation  exercises  were 
conducted  for  the  purpose  of  testing  the  Virtual  Surveillance  Target  Attack  Radar  Simulation 
(VSTARS).  The  protocol  data  units  (PDUs)  broadcast  by  Janus  and  TAFSM  during  these 
simulation  exercises  were  analyzed  using  the  JADS  Toolbox.  All  PDUs  were  found  to  be 
compliant  with  DIS  standards  as  modified  by  the  ETE  Test. 

The  DIS  standard  requires  that  for  an  entity  state  PDU  (ESPDU),  the  "...  timestamp  shall  be 
specified  using  a  32-bit  unsigned  integer  representing  units  of  time  passed  since  the  beginning  of 
the  current  hour.  The  least  significant  bit  shall  indicate  whether  the  timestamp  is  absolute  or 
relative.”  Because  the  ETE  Test  vignettes  used  in  Janus  involve  nearly  ten  thousand  entities  and 
run  for  nearly  seven  hours,  it  was  necessary  to  modify  the  standard  and  represent  time  as  a  32-bit 
unsigned  integer  representing  milliseconds  since  the  beginning  of  the  vignette  (relative 
timestamp).  This  allows  the  tester  to  trace  an  ESPDU  back  to  a  discrete  event  that  occurred 
within  the  Janus  vignette.  This  is  a  common  practice  in  most  DIS  SE  that  exist  for  periods 
greater  than  one  hour. 

TAFSM  also  uses  a  relative  timestamp  in  the  PDUs  that  it  generates.  Within  TAFSM,  the 
timestamp  represents  milliseconds  since  midnight  of  the  day  of  the  test.  This  method  is  the  same 
as  that  used  by  the  JADS  logger  and  allows  time  tracking  of  the  PDU  throughout  the  ETE  Test 
SE. 


4.2  Perforin  Conceptual  Model  Verification  (Step  3) 

The  conceptual  model  verification  was  performed  during  the  development  of  the  feasibility  study. 
At  this  time,  several  conceptual  models  were  verified  by  inspection  and  one  was  chosen  for 
further  development  into  the  ETE  Test  SE. 
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The  conceptual  models  developed  for  the  ETE  Test  SE  consisted  of  a  series  of  as  yet  undefined 
simulation  nodes  that  required  inputs  that  would  be  used  to  generate  outputs.  The  outputs  of  one 
or  more  simulation  nodes  would  than  become  the  inputs  for  another  simulation  node  in  the  SE. 
The  conceptual  model  developed  for  the  ETE  Test  SE  is  represented  by  Figure  3. 


Figure  3.  ETE  Test  SE  Conceptual  Model. 

As  can  be  seen  from  the  figure,  this  simple  conceptual  model  of  the  ETE  Test  SE  calls  for  a 
source  of  information  about  potential  targets  (massive  war  game)  that  would  be  provided  to  a 
radar  simulation  representing  the  radar  onboard  the  Joint  Surveillance  Target  Attack  Radar 
System  (Joint  STARS)  E-8C  aircraft.  This  radar  simulation  would  provide  radar  output  to  an 
intelligence  gathering,  analysis,  and  targeting  cell  that  would  than  use  the  radar  output  to 
nominate  targets  for  engagement  by  the  shooter  within  the  SE.  The  shooter  would  then  engage 
the  nominated  targets  by  interacting  with  the  massive  war  game.  The  massive  war  game  would 
determine  the  effect  of  the  shots  fired  and  would  in  turn  report  changes  in  target  status  to  the 
radar  simulation  completing  the  ETE  Test  cycle. 

4.3  Perforin  Architectural  Design  Verification  (Step  4) 

The  architectural  design  verification  was  performed  during  the  development  of  the  PTP.  It 
consisted  of  a  document  review,  the  development  of  a  preliminary  model  of  the  SE,  an  allocation 
of  functions  and  capabilities  to  the  nodes  of  the  SE,  operational  requirements  mapping,  a 
determination  of  interface  requirements  and  database  requirements,  and  initial  personnel  and  test 
requirements. 
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4.3.1  Preliminary  Model  of  End-to-End  Synthetic  Environment 

The  preliminary  model  of  the  ETE  Test  SE,  Figure  3-15  in  the  Joint  Advanced  Distributed 
Simulation  Joint  Test  and  Evaluation  Program  Test  Plan,  February  1996,  is  shown  here  as  Figure 
4. 


Janus  Constructive  Model 


ACE  =  analysis  and  control  element  ASAS  =  All  Source  Analysis  System  SCDL  =  surveillance  control  data  link 


Figure  4.  Preliminary  Model  of  the  ETE  Test  SE 


4.3.2  Allocation  of  Functions 

The  allocation  of  functions  for  the  preliminary  model  of  ETE  Test  SE  is  shown  in  Table  1. 
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Table  1.  Allocation  of  Functions  &Capabilities  for  the  Preliminary  Model  of  ETE  Test  SE 


NODE 

FUNCTION 

CAPABILITY 

Virtual  E-8C 

Provides  radar  reports  to  SE 

Receives  ESPDUs  and 
provides  moving  target 
indicator  (MTI)  radar  (SAR) 
radar  reports  to  GSM  via  the 
SCDL. 

Ground  Station  Module 
(GSM) 

Provides  target  information  to 
analysis  and  control  element 
(ACE)/A11  Source  Analysis 
System  (ASAS) 

Receives  MTI  and  SAR  radar 
reports  via  the  SCDL  and 
provides  spot  reports  to 
ACE/ASAS. 

Virtual  Unmanned  Aerial 
Vehicle  (UAV) 

Provides  real-time  visual 
imagery  of  battlefield  to  GSM 

Receives  ESPDUs  and 
provides  visual  imagery  to 

GSM 

Test  Control  and  Analysis 
Center 

Controls  test  and  conducts 
both  real-time  and  post-test 
analysis  of  data 

Receives  and  logs  all  PDU  and 
message  traffic,  except  for 
SCDL. 

Virtual  Army  Tactical  Missile 
System  (ATACMS) 

Fires  virtual  ATACMS 
missiles  into  Janus 

Receives  fire  missions  from 
Virtual  Fire  Direction  Center 
and  fires  ATACMS  missile  at 
designated  target. 

Virtual  Fire  Direction  Center 

Issues  fire  missions  to  Virtual 
ATACMS 

Receives  target  nominations 
from  ACE/ASAS  and  issues 
fire  missions  to  virtual 

ATACMS 

Virtual  ACE/ASAS 

Analyzes  available  intelligence 
and  nominates  targets  for 
engagement 

Receives  target  information 
from  GSM  and  imagery  from 
UAV.  Nominates  targets 
based  on  analysis  of  available 
intelligence. 

Janus  Constructive  Model 

Massive  war  game  of  at  least 
5000  entities. 

Executes  preexisting  scenario 
with  operator  interaction; 
determines  effects  of 

ATACMS  hits;  and  broadcasts 
ESPDUs  reflecting  status  of 
each  entity  within  the  war 
game. 

4.3.3  Operational  Requirements  Mapping 

The  operational  requirements  mapping  was  performed  during  the  development  of  the  PTP  and  is 
in  paragraph  3.4  of  the  Joint  Advanced  Distributed  Simulation  Joint  Test  and  Evaluation  Program 
Test  Plan,  February  1996.  An  example  is  shown  below. 
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JADS  Joint  Test  Force 

-  Overall  responsibility  for  the  planning,  execution,  analysis,  and  reporting  of  the  test. 

-  Develops  advanced  distributed  simulation  (ADS)  measures  and  related  events. 

-  Analyzes  and  evaluates  ADS  measures. 

-  Reports  interim  and  final  results  to  the  Office  of  the  Secretary  of  Defense  (OSD). 

-  Develops  the  Phase  2  test  plan. 

-  Develops  V&V  plan  for  ETE  Phase  2. 

-  Establishes  necessary  communication  links  with  test  participants. 

-  Activates  ETE  Test  DIS  network. 

-  Develops  necessary  scenarios  for  ETE  Test  Phase  2. 

4.3.4  Determination  of  Interface  Requirements 

The  interface  and  database  requirements  for  the  preliminary  model  of  the  ETE  Test  SE  are  shown 
in  Table  2. 

Table  2.  Interface  &d  Database  Requirements  for  the  Preliminary  Model  of  ETE  Test  SE 


NODE 

INTERFACE 

REQUIREMENTS 

DATABASE 

REQUIREMENTS 

Virtual  E-8C 

DIS  network  interface  unit, 
surveillance  control  data  link 
(SCDL)  1553  interface  unit 

Enumeration  database 

System  databases 

Ground  Station  Module 
(GSM) 

SCDL  1553  interface  unit, 
system  communication 
interfaces 

System  databases 

Virtual  unmanned  aerial 
vehicle  (UAV) 

DIS  network  interface  unit, 
system  communication 
interfaces 

Enumeration  database 

Terrain  databases 

Test  Control  and  Analysis 
Center 

DIS  network  interface  unit 

Enumeration  database 

Terrain  database 

Virtual  Army  Tactical  Missile 
System  (ATACMS) 

DIS  network  interface  unit 

Enumeration  database 

Terrain  databases 

System  databases 

Virtual  Fire  Direction  Center 

DIS  network  interface  unit 

System  databases 

Virtual  analysis  and  control 
element  (ACE)/A11  Source 
Analysis  System  (ASAS) 

System  communication 
interfaces 

System  databases 
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Janus  Constructive  Model 

DIS  network  interface  unit 

Terrain  databases 

Force  databases 

Enumeration  databases 

Scenarios 

4.3.5  Initial  Personnel  and  Test  Requirements 

Initial  personnel  and  test  requirements  are  in  paragraph  3.4  of  the  Joint  Advanced  Distributed 
Simulation  Joint  Test  and  Evaluation  Program  Test  Plan,  February  1996. 

4.4  Perform  Detailed  Design  Verification  (Step  5) 

The  initial  detailed  design  verification  was  performed  by  the  ETE  Test  V&V  team  during  Phase  1 
of  the  ETE  Test.  The  V&V  team: 

a)  reviewed  model  and  simulation  (M&S)  component  documentation  and,  if  necessary, 
source  code  to  determine  component  ability  to  perform  their  assigned  functions; 

b)  executed  key  algorithms  to  ensure  they  functioned  appropriately  to  address  the 
exercise  requirements; 

c)  assessed  the  logic  of  the  proposed  interconnections  of  the  components  by  evaluating 
the  proposed  interchange  of  PDUs;  and, 

d)  analyzed  the  exercise  design  for  its  rigor. 

In  addition,  members  of  the  V&V  team  evaluated  the  appropriateness  and  sufficiency  of  the  input 
data  selected  for  use  in  the  exercise. 

This  activity  involved  five  major  tasks:  evaluate  detailed  design,  evaluate  interface  design,  verify 
data  and  databases,  evaluate  V&V  test  plans,  and  evaluate  training  requirements. 

4.4.1  Evaluate  Detailed  Design 

This  task  involves  determining  if  the  design  is  sufficient  to  ensure 

a)  the  individual  M&S  components  are  capable  of  representing  the  exercise 
phenomenology  at  appropriate  levels  of  resolution;  and 

b)  the  underlying  network  assets  can  support  the  exchange  data  among  the  components 
at  the  necessary  levels  of  fidelity. 

This  task  was  accomplished  in  the  following  manner: 

a)  Acceptance  testing  of  M&S  components  to  ensure  they  are  capable  of  representing  the 
exercise  phenomenology  at  appropriate  levels  of  resolution. 

b)  Characterization  of  underlying  network  assets  to  determine  if  they  can  support  the 
exchange  of  data  among  the  components  at  the  necessary  levels  of  fidelity. 
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c)  Simulation  of  elements  of  the  detailed  design  to  determine  their  functionality  using 
POWERSIM™  Version  2.02. 
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4. 4. 1.1  A cceptance  Testing 


Acceptance  testing  was  conducted  on  VSTARS,  Advanced  Radar  Imaging  Emulation  System 
(ARIES),  and  the  JADS  ground  data  terminal  1553  bus  interface  unit.  Results  of  the  acceptance 
tests  may  be  found  at 

a)  Appendix  A,  JADS  Ground  Data  Terminal  1553  Bus  Interface  Unit  Acceptance  Test 

b)  Appendix  B,  Advanced  Radar  Imaging  Emulation  System  (ARIES)  Acceptance  Test 

c)  Appendix  C,  VSTARS  Acceptance  Test 

4. 4. 1.2  Characterization  of  Network  Assets 

Characterization  of  the  ETE  Test  network  assets  was  conducted  by  the  JADS  Network  and 
Engineering  (N&E)  team.  Results  of  the  characterizations  may  be  found  at 

a)  Appendix  D,  Characterization  Reports  From  JADS  JTF/ETE  (N&E) 

b)  Appendix  E,  LWX  and  Cisco  Router  Performance  Test  Report 

4. 4. 1.3  Simulation  Using  POWERSIM1™ 

Early  in  Phase  1 ,  prior  to  the  establishment  of  the  ETE  Test  network,  a  simulation  was  built  using 
POWERSIM™  to  determine  if  the  detailed  design  could  support  the  desired  SE.  A  model  of  the 
portion  of  the  network  that  would  handle  ESPDUs  was  developed  and  the  traffic  over  the 
network  was  studied  using  various  values  and  distributions  of  latency,  PDU  dropout,  etc. 

In  addition  to  changing  network  variables,  such  as  latency,  the  load  on  the  network  could  be 
increased  by  varying  the  number  of  entities,  the  number  of  moving  entities,  and  other  similar 
variables.  Finally,  the  DIS  parameters,  such  as  heartbeat  rate,  could  be  varied  at  will.  This 
resulted  in  a  very  robust  simulation. 

This  simulation  was  able  to  verify  that  the  detailed  design  was  capable  of  supporting  a  wide  range 
of  network  conditions  and  should  be  able  to  support  the  ETE  Test  SE.  In  addition,  the  simulation 
provided  design  information  to  the  engineers  who  were  developing  the  VSTARS  network 
interface  unit. 

Some  of  the  POWERSIM™  output  is  included  as  Appendix  F,  POWERSIM™. 

4.4.2  Evaluate  Interface  Design 

This  task  was  performed  during  the  acceptance  testing  of  the  components  of  the  SE  and  evaluated 
the  ability  of  the  individual  M&S  components  to  interoperate  with  each  other  and  with  the 
network  by 
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a)  determining  that  interfaces  among  components  and  interfaces  with  the  synthetic 
environment  are  adequate  and  sufficient  to  allow  consistency  in  the  level  of  details, 
data  fidelity,  and  data  sources,  and  sufficient  modes  of  operation; 

b)  ensuring  that  user  interfaces  for  input  and  output  can  pass  information  to  accomplish 
efficient  scenario  construction,  component  execution,  network  management,  and 
report  generation,  and 

c)  evaluating  the  impact  of  network  factors  such  as  latency  produced,  network  loading, 
and  filtering  requirements. 

Appendices  A  through  E  contain  the  results  of  the  acceptance  tests  and  network  investigations. 

4.4.3  Verify  Data  and  Databases 

There  are  two  primary  sets  of  simulation  data  used  within  the  ETE  Test.  They  are  terrain  data 
used  by  VSTARS,  Janus,  and  TAFSM,  and  weapon  system  performance  data  used  by  Janus  and 
TAFSM. 

Due  to  the  unclassified  nature  of  the  test,  it  was  decided  that  the  unclassified  system  performance 
data  provided  for  use  within  Janus  and  TAFSM  would  be  used  for  the  ETE  Test.  This  is  possible 
because  the  moving  target  indicator  (MTI)  radar  simulation  of  VSTARS  detects  targets  based 
upon  their  velocity.  The  range  of  velocities  represented  by  the  unclassified  system  performance 
data  is  the  same  as  would  be  expected  from  the  actual  target  during  normal  operations. 

Unfortunately,  the  terrain  data  available  for  use  within  the  ETE  Test  were  unusable  because  of 
their  low  fidelity  and  resolution.  It  was  therefore  decided  to  construct  our  own  terrain  databases. 
These  terrain  databases  will  be  verified  during  Phase  2. 

4.4.4  Evaluate  V&V  Test  Plans 

During  Phase  1,  the  V&V  test  plan,  ETE  Test  Activity  Plan,  Appendix  C,  Verification  and 
Validation  Plan  for  the  End-To-End  (ETE)  Test,  was  published  and  reviewed  by  the  JADS  ETE 
Test  accreditation  board. 

4.4.5  Evaluate  Training  Requirements 

An  evaluation  of  training  requirements  revealed  that  all  of  the  real  systems  used  within  the  ETE 
Test  are  fielded  systems  with  operators  who  have  attended  the  appropriate  training  schools.  No 
new  equipment  training  is  required  pursuant  to  the  ETE  Test.  It  was  observed,  however,  that 
many  of  the  operators  had  either  not  been  well  trained  during  their  initial  entry  courses  or  had 
forgotten  certain  skills  from  lack  of  use  or  lack  of  training  opportunities.  As  an  example,  ground 
station  module  (GSM)  operators  receive  no  training  in  the  use  of  Joint  STARS  synthetic  aperture 
radar  (SAR)  images  during  their  initial  entry  course. 


This  situation  requires  that  during  the  functionality  and  integration  tests  and  risk  reduction  tests 
scheduled  for  Phase  2,  the  operators  of  the  fielded  systems  will  need  to  be  retrained  on  the  use  of 
their  equipment. 

The  existing  simulations,  Janus  and  TAFSM,  require  minimal  training  of  their  operators  on  newly 
added  features.  This  training  was  conducted  as  new  features  were  added  during  Phase  1 . 

VSTARS  utilizes  the  standard  operator  interface  used  by  Joint  STARS.  As  such,  it  requires  no 
new  equipment  training  of  the  operators.  Because  it  is  an  emulation  of  the  radar  subsystem  used 
by  Joint  STARS,  it  may  be  used  for  refresher  training  as  mentioned  above. 

5.  Validation  Tasks 

5.1  Perform  Conceptual  Model  Validation  (Step  3) 

The  JADS  ETE  Test  conceptual  model  validation  was  performed  during  the  conduct  of  the 
feasibility  study.  During  this  period,  discussions  were  held  with  the  Military  Intelligence  School, 
Fort  Huachucha,  Arizona;  the  Field  Artillery  School,  Fort  Sill,  Oklahoma;  Joint  STARS  Program 
Office  and  Northrop  Grumman,  Melbourne,  Florida;  and  the  U.S.  Army  Training  and  Doctrine 
Command  (TRADOC)  Analysis  Center,  White  Sands  Missile  Range,  New  Mexico,  as  to  the 
validity  of  the  conceptual  model. 

All  agreed  that  doctrinally  and  functionally,  the  conceptual  model  represented  the  real-life  target 
detection,  target  identification,  target  assignment,  target  engagement,  battle  damage  assessment 
loop  that  the  ETE  Test  desired  to  replicate.  The  conceptual  model  considered  during  this 
validation  step  is  shown  in  Figure  1. 

6.  Conclusion 

This  verification  and  validation  report  covers  the  first  five  steps  of  the  exercise  W&A  process  as 
referred  to  in  the  Recommended  Practice  for  Distributed  Interactive  Simulation— Verification, 
Validation,  and  Accreditation,  4  November  1996.  This  process,  commonly  referred  to  as  the  DIS 
Nine  Step  W&A  Process,  was  developed  by  the  DIS  Verification  ,  Validation  and  Accreditation 
subgroup  of  the  Workshops  on  Standards  for  the  Interoperability  of  Distributed  Simulations,  of 
which  the  author  of  this  report  is  a  member. 

No  final  conclusions  may  be  inferred  from  this  report  as  to  the  authenticity  and  validity  of  the 
ETE  Test  synthetic  environment.  What  may  be  determined  is  that  these  early  V&V  steps 
provided  confidence  in  the  design  of  the  environment  and  were  instrumental  in  developing 
specifications  and  requirements  for  the  continued  development  of  the  environment.  Additionally, 
problems  were  identified  early  in  the  program,  as  a  result  of  these  V&V  efforts.  This  enabled  the 
ETE  Test  team  to  react  proactively  manner  throughout  the  development  of  the  ETE  Test 
synthetic  environment. 
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Additionally,  this  report  helps  to  authenticate  the  DIS  Nine  Step  W&A  Process  as  being  a 
feasible  process  that  may  be  adapted  to  support  the  V&V  of  a  DIS  synthetic  test  environment 
involving  live  elements,  virtual  simulations,  and  constructive  simulations. 
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Appendix  A 

JADS  Ground  Data  Terminal  1553  Bus  Interface  Unit  Acceptance  Test 


JADS  Bus  Interface  Unit  (BIU)  Qualification  Test  Procedure 

Three  test  configurations  are  identified  herein  to  support  the  test  levels  and  definitions  described 
by  the  previously  submitted  software  test  plan  (STP).  This  procedure  describes  each  test 
configuration,  tests  conducted  for  that  configuration,  and  a  brief  procedure  for  each  test.  All  test 
definitions  from  the  software  test  plan  are  included  within  this  procedure;  however,  they  will  be 
executed  in  accordance  with  the  test  configuration  order  presented  below. 

Test  Configuration  I 

Software  emulation  of  common  ground  station  (CGS)  running  on  JADS  BIU  workstation 

JADS  BIU  software  running  on  JADS  BIU  workstation 

Software  emulation  of  T-l  link  running  on  “Kong”  workstation 

Triax  termination  on  primary  1553  connector  which  links  the  CGS  emulation  software  to  the 
JADS  BIU  software  via  1553  bus. 

Ethernet  link  between  JADS  BIU  and  “Kong”  network 

When  each  of  the  three  software  components  have  been  initiated,  the  CGS  emulator  will  send 
control  messages  and  uplink  messages  (  rate  =  0.1  hertz  [Hz])  to  the  BIU  over  the  1553  bus, 
automatically  triggering  the  ground  data  terminal  (GDT)  status  transitions.  The  T-l  link  emulator 
software,  upon  receiving  the  first  uplink  message  from  the  BIU  over  the  Ethernet  link,  will 
respond  by  sending  downlink  messages  (  rate  =  80  Hz)  and  aircraft  location  messages  ( rate  =  1 
Hz).  When  the  JADS  BIU  software  transitions  to  the  downlink  enabled  mode,  it  will  pass  the 
messages  through  the  1553  bus  where  they  will  be  detected  by  the  CGS  emulator  software. 
Results  of  the  tests  will  be  verified  by  examination  of  log  files  generated  as  the  above  scenario  is 
executed. 

Test  Configuration  I  Coverage 

a)  GDT  Initial  Status  (STP  Paragraph  4. 1.4.1) 

Verify  control  word  portion  of  initial  GDT  status  report  from  CGS  emulator  log  file 

b)  GDT  Initial  Status  Message  Blocking  (STP  Paragraph  4. 1.4.2) 


Examine  JADS  BIU  log  file  to  verify  that  uplink  messages  are  counted  only  when  uplink  is 
enabled,  and  that  downlink  messages  are  counted  only  when  downlink  is  enabled. 

c)  GDT  Initial  Status  Transition  (STP  Paragraph  4. 1.4.3) 

Examine  JADS  BIU  log  file  to  verify  that  uplink  Enabled  and  downlink  enabled  mode  transitions 
take  place. 

d)  GDT  Uplink  Messages  (STP  Paragraph  4. 1.4.4) 

Examine  T-l  emulator  log  file  to  verify  the  indication  of  test  uplink  messages  passed  through  the 
JADS  BIU  and  transmitted  via  Ethernet  to  the  “Kong”  workstation. 

e)  GDT  Uplink  Message  Latency  (STP  Paragraph  4. 1.4.5) 

Examine  JADS  BIU  log  file  to  record  maximum  and  average  uplink  latency  values  in  milliseconds. 

f)  GDT  Downlink  Messages  (STP  Paragraph  4. 1.4.6) 

Examine  JADS  BIU  log  file  to  record  maximum  and  average  uplink  latency  values  in  milliseconds. 

g)  GDT  Aircraft  Location  Messages  (STP  Paragraph  4. 1.4.7) 

Examine  CGS  emulator  log  files  to  verify  the  indication  of  aircraft  location  data  updating  the 
appropriate  fields  of  the  GDT  status  message. 

Test  Configuration  I  Results 

All  elements  of  test  configuration  I  meet  requirements. 

Test  Configuration  II 

CGS  server  1553  connection  to  provide  downlink  data  request 

JADS  BIU  software  running  on  JADS  BIU  workstation 

Software  emulation  of  T-l  link  running  on  “Kong”  workstation 

Cable  connection  between  primary  1553  connector  which  links  the  CGS  server  to  the  JADS  BIU 
software  via  1553  bus. 

Ethernet  link  between  JADS  BIU  and  “Kong”  network 

Requests  for  downlink  data  are  generated  periodically  (20  Hz)  by  the  CGS  server.  The  T-l  link 
software  emulation  generates  aircraft  location  and  downlink  messages  at  a  rate  which  is 


comparable  to  that  supported  by  the  actual  T-l  link  maximum.  Uplink  messages  will  be  generated 
from  the  CGS  workstation  console,  and  when  the  first  message  is  passed  through  the  JADS  BIU 
to  the  T-l  emulation  software,  downlink  and  aircraft  location  message  transmission  will  begin. 
Results  of  the  tests  will  be  verified  by  log  files  generated  on  the  JADS  BIU  workstation. 

Test  Configuration  II  Coverage 

a)  GDT  Downlink  Latency  (STP  Paragraph  4. 1.4.7) 

Examine  JADS  BIU  log  file  to  record  maximum  and  average  downlink  latency  values  in 
milliseconds. 

b)  GDT  Traffic  Capacity  (STP  Paragraph  4. 1.4.9) 

Examine  JADS  BIU  log  file  to  record  maximum  throughput  rate  in  messages  per  second. 

Test  Configuration  II  Results 

All  elements  of  test  configuration  II  meet  requirements. 

Test  Configuration  III 

CGS  Server  1553  connection  to  JADS  BIU  workstation 

E8  aircraft  simulator  connection  to  JADS  BIU  workstation  (  T-l /  Integrated  Digital  Network 
Exchange  [IDNX]) 

The  workstation  will  require  reconfiguration  and  reboot  to  incorporate  new  internet  protocol  (IP) 
address  and  port  data,  which  are  necessary  to  support  its  operational  configuration. 

Test  Configuration  III  Coverage 

a)  GDT  Initial  Status  (STP  Paragraph  4.1.4.10) 

Capture  GDT  status  report  at  CGS  console  using  bus  monitor  log  facility. 

b)  GDT  Initial  Status  Transition  (STP  Paragraph  4. 1.4.1 1) 

Use  CGS  manage  links  controls  and  periodic  status  displays  on  JADS  BIU  console  to  verify  the 
status  transitions. 

c)  GDT  Aircraft  Location  Messages  (STP  Paragraph  4.1.4.14) 

Use  CGS  workstation  imagery  window  to  display  the  simulated  E  -  8C  flight  path. 


d)  GDT  Uplink  Messages  (STP  Paragraph  4.1.4.12) 

Generate  radar  service  request  (RSR)  and  freetext  messages  using  CGS  workstation  and  confirm 
receipt  at  Grumman  facility. 

e)  GDT  Downlink  Messages  (STP  Paragraph  4.1.4.12) 

Use  CGS  message  alert  facility  to  display  freetext  message  originated  from  the  simulated  E  -  8C; 
use  imagery  window  to  display  moving  target  indicator  (MTI)  and  synthetic  aperture  radar  (SAR) 
imagery  patterns. 

Test  Configuration  III  Results 

a)  GDT  Uplink  Messages  (STP  Paragraph  4.1.4.12) 

Generate  RSR  and  freetext  messages  using  CGS  workstation  and  confirm  receipt  at  Grumman 
facility. 

Capability  not  present  on  CGS  workstation  used  for  acceptance  test.  Requirement  deferred  until 
installation  of  BIU  on  light  ground  support  module  (LGSM)  at  Fort  Hood. 

b)  GDT  Downlink  Messages  (STP  Paragraph  4.1.4.12) 

Use  CGS  message  alert  facility  to  display  freetext  message  originated  from  the  simulated  E  -  8C; 
use  imagery  window  to  display  MTI  and  SAR  imagery  patterns. 

Image  dropouts  observed  when  displaying  MTI  and  SAR  imagery  patterns.  This  is  believed  to  be 
caused  by  too  fast  a  data  transmission  rate  when  transmitting  MTI  and  SAR  imagery  patterns. 
We  will  modify  Virtual  Surveillance  Target  Attack  Radar  System  (VSTARS)  surveillance  control 
data  link  (SCDL)  interface  to  make  data  transmission  rate  adjustable  and  tune  data  transmission 
rate  during  installation  of  BIU  at  Fort  Hood. 


Appendix  B 

Advanced  Radar  Imaging  Emulation  System  (ARIES)  Acceptance  Test 

Section  5  extracted  from  the  Final  Report  for  the  Advanced  Radar  Imaging  Emidation  System 
(AIRES)  Program. 

5.  Factory  Acceptance  Testing 

A  test  suite  of  images  has  been  established  as  the  baseline  regression  and  factory  acceptance  test 
for  the  ARIES  product.  Each  ARIES  requirement  is  mapped  to  at  least  one  test  image  which 
demonstrates  ARIES  ability  to  support  that  requirement.  Additional  test  cases  are  provided  to 
validate  system  performance  and  accuracy. 

5.1  Requirements  versus  Test  Cases 

Table  5.1-1  defines  all  ARIES  requirements  and  maps  each  to  a  test  case.  The  test  cases  are 
described  in  the  table  that  follows.  All  test  cases  have  been  run  and  the  images,  FID  (Feature 
Identification)  maps,  shadow  maps,  gain  maps  (based  on  geometry  and  reflectivity  of  the  surface) 
and  other  output  information  examined  to  verily  accuracy. 


Table  5.1-1  ARIES  Requirements 


Req  # 

Test  # 

Requirements 

100 

1 

ARIES  will  simulate  a  JSTARS  SAR  image  in  real-time  on  a  DEC  Alpha 
600/533  workstation  as  specified  in  the  Classified  Appendix 

110 

2 

ARIES  shall  implement  an  earth  curvature  corrected  coordinate  system. 

120 

3 

ARIES  shall  simulate  a  fixed  image  size  with  dimensions  of  2  km  x  4  km. 

130 

3 

ARIES  shall  simulate  an  image  with  constant  resolution  in  range  and 
azimuth. 

140 

4 

ARIES  shall  simulate  images  in  the  slant  plane. 

150 

4 

ARIES  shall  generate  a  magnitude  simulated  image. 

160 

4 

ARIES  shall  simulate  leading  edge  effects. 

170 

ARIES  shall  simulate  the  effects  of  range  layover. 

190 

2 

ARIES  shall  simulate  the  effects  of  obscuration  along  the  sensor  line-of- 
sight. 

200 

4 

ARIES  shall  simulate  a  single  look  capability. 

210 

4 

ARIES  shall  simulate  a  focused  Spot  SAR. 

220 

4 

ARIES  shall  simulate  depression  angles  up  to  12  degrees  relative  to 
horizon. 

230 

4 

ARIES  shall  simulate  a  maximum  squint  angle  of  +/-  60  degrees  off 
broadside. 

240 

ARIES  shall  simulate  summer  seasonal  conditions. 
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250 

ARIES  shall  simulate  clear  atmospheric  conditions. 

260 

6 

ARIES  shall  simulate  calm  wind  conditions. 

270 

4 

ARIES  shall  simulate  SAR  imagery  in  accordance  with  JSTARS  sensor 
performance  parameters. 

280 

4 

ARIES  shall  simulate  X-band. 

290 

4 

ARIES  shall  simulate  HH  polarization. 

300 

3 

ARIES  shall  simulate  a  single  resolution  in  range  and  azimuth  as  specified 
in  the  classified  appendix. 

320 

2 

ARIES  shall  use  an  elevation  database  for  simulating  topographical 
effects  within  a  simulated  image. 

325 

8 

ARIES  shall  support  an  elevation  database  which  encompasses  an  area  up 
to  512  km  x  512  km. 

330 

4 

ARIES  shall  use  a  feature  database  to  locate  natural  and  cultural  features 
within  a  simulated  image. 

340 

2 

ARIES  shall  accept  earth  curvature  corrected  elevation  data  from  the 

GFE  database. 

350 

2 

ARIES  shall  accept  earth  curvature  corrected  feature  data  from  the  GFE 
database. 

360 

4 

ARIES  shall  accept  point  feature  data. 

370 

4 

ARIES  shall  accept  linear  feature  data. 

380 

26 

ARIES  shall  accept  areal  feature  data. 

390 

n/a 

ARIES  shall  incorporate  effects  of  features  and  topography  which  lie 
outside  of  Area  of  Interest  (AOI)  that  influence  obscuration  within  the 
AOI. 

420 

8 

Rocky  flats  (maximum  boulder  size  of  1  foot) 

430 

9 

Packed  sand  and  gravel 

440 

4 

Loose  sand 

450 

10 

Windblown  dunes 

460 

11 

Dry  depressions  with  sandy  bottoms 

470 

12 

Wadis 

480 

13 

Escarpments 

490 

14 

Salt  marshes 

500 

15 

Salt  flats 

510 

16 

Flood  Plains 

520 

4 

Date  palm  orchards 

530 

17 

Irrigated  fields 

540 

4 

Buildings 

550 

4 

Cylindrical  storage  tanks 

560 

4 

Oil  wells 

570 

2 

Above  ground  pipelines 

580 

18 

Soil/sand/dirt  berms  referred  to  as  revetments 

590 

19 

Below  ground  sand/dirt  trenches 

600 

4 

Above  ground  concrete  and  dirt  bunkers 
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610 

19 

Sand/dirt  ditches 

620 

4 

Paved  roads 

630 

20 

Unpaved  roads 

640 

21 

Loose  dirt  trails  (less  than  1  road  width) 

650 

2 

Railroad  tracks 

660 

4 

Airfields  -  composed  of  buildings  and  runways 

670 

4 

Transmission  towers  -  4  sided  pyramidal 

680 

4 

Electric  power  lines 

690 

22 

Dirt  and  concrete  dikes  and  levees 

700 

23 

Dirt  and  concrete  walls  and  berms 

710 

19 

Anti-tank  ditches 

720 

2 

Pipelines  within  trenches 

730 

24 

Quarries 

740 

2 

Chain  link  fences 

750 

2 

Concertina  fences 

760 

2 

Barbed  wire  fences 

770 

2 

Bridges 

800 

4 

ARIES  shall  simulate  moving  targets. 

810 

4 

ARIES  shall  simulate  stationary  targets. 

830 

4 

Tanks,  ie.  T-72,  T-72  w/plow,  T-72  w/mine  roller,  T-54/55 

840 

4 

Air  Defense  Systems,  ie.  Long  Track  Radar,  MTLB,  ZSU-23-4 

850 

4 

Armored  Personal  Carrier,  ie.  BMP-1,  BMP-2,  MTLB 

860 

4 

Artillery,  i.e.  2S1,  2S3 

870 

4 

Armored  Cars,  ie.  BRDM  w/machine  guns,  BRDM  w/AT-5s,  BTR-60, 
BTR  w/120mm  mortars. 

880 

4 

Artillery,  i.e.  BM-21,  BM-22,  D-30,  T-12 

890 

4 

Support  Trucks,  i.e.  UAZ-469,  5-Ton  CAC,  5 -ton  fuel,  2  'A-ton  cargo 

900 

4 

MI-24 

910 

4 

MI-8  Hip 

920 

4 

HAVOC 

930 

4 

ARIES  shall  support  TEL  targets. 

940 

b2.1 

ARIES  shall  simulate  SAR  imagery  in  accordance  with  data  provided 
through  an  interface  with  Northrop  Grumman's  RPSI  as  specified  in  the 
ARIES-RPSI  Interface  Control  Document. 

950 

b2.1 

All  coordinates  passed  to  ARIES  through  the  RPSI  shall  be  relative  to  a 
topocentric  coordinate. 

960 

b2.1 

Area  of  Interest  Centeroid  (x,y,z)  (m) 

970 

b2.1 

Resolution  (m) 

980 

b2.1 

Wavelength  (m) 

990 

b2.1 

Dwell  Time  (s) 

1000 

b2.1 

Azimuth  extent  (radians) 

1010 

b2.1 

Range  extent  (m) 

1020 

b2.1 

Sensor  position  (x,y,z)  (m) 

1030 

b2.1 

Sensor  velocity  vector  (m/s) 
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1040 

b2.1 

1050 

b2.1 

1060 

b2.1 

1070 

b2.1 

1080 

b2.1 

1090 

b2.1 

1100 

b2.1 

1110 

b2.1 

1120 

1170 

b2.1 

1180 

b2.1 

1190 

b2.1 

1200 

1 

1210 

b2.1 

1220 

1230 

b2.1 

1260 

b2.1 

1300 

b2.1 

1310 

b2.1 

Antenna  orientation  vector 
Number  of  targets  within  image 

For  each  target  in  the  image  area,  Target  category,  ie.  tank,  truck, 
launcher 

For  each  target  in  the  image  area.  Target  subcategory,  ie.  M-l,  M-60 

For  each  target  in  the  image  area,  Target  specifics 

For  each  target  in  the  image  area,  Target  position  centroid  (x,y,z)  (m) 

For  each  target  in  the  image  area,  Target  velocity  vector  (m) 

For  each  target  in  the  image  area,  Target  orientation  vector 

For  each  target  in  the  image  area,  target  appearance 

ARIES  software  shall  execute  on  the  Alpha  600  5/333  workstation. 

LM  shall  have  1GB  of  RAM  on  the  Alpha  600  5/333  dedicated  to  ARIES 
during  simulation  activity. 

ARIES  executable  software  and  resident  operating  system  shall  have 
access  to  100%  of  the  processing  resource  CPU  time  on  the  Alpha  600 
5/333  during  simulation  activity. 

ARIES  software  shall  execute  under  OpenVMS,  version  6.2  or  later. 
ARIES  software  shall  be  compatible  with  the  RPSI  environment. 

ARIES  software  shall  return  a  pass  status  upon  successful  completion  of 
image  generation. 

ARIES  software  shall  return  a  fail  status  upon  unsucessful  completion  of 
image  generation. 

ARIES  shall  pass  image  data  to  the  RPSI  with  2000/R  pixels  in  cross¬ 
range  by  4000/R  pixels  in  range  where  R  is  the  resolution  parameters  in 
meters  passed  by  the  RPSI. 

ARIES  software  shall  accept  a  Ground  Reference  Coverage  Area 
(GRCA)  as  input  from  the  RPSI  as  specified  in  the  ARIES  -  RPSI 
Interface  Control  Document. 

ARIES  shall  simulate  an  image  which  is  completely  encompassed  by  the 
GRCA  boundaries. 


5.2  Test  Cases  Defined 

Tables  5.2-1  defines  the  test  cases  and  describes  what  each  test  case  involves. 

Table  5.2-1  Test  Care  Description 
Test  Title  Description 

# 

1  Alpha  Execution  Run  alpha  test  program  on  both  the  SUN  and  the  alpha 

with  default  values  to  generate  an  image  output  file. 
Verify  the  image  from  the  alpha  matches  the  same  image 
generated  on  the  SUN.  This  image  is  generated  using  the 
small  library,  small  DFAD  and  small  GRCA. 


2 


3 

4 


5 

6 

7 

8 


9 

10 

11 

12 

13 

14 

15 

16 

17 

18 


19 

20 
21 


22 


Terrain  /  Shadow  / 
Feature  Verfication 


Resolution 
Distortion 
General  Image 
Quality 


Terrain  /  Feature 

Matching 

Windy 

GRCA  Test 

Coarse  +  Smooth 
Textures  +  Stream  + 
Mnt. 

Fine  +  Smooth 
Textures 
Dunes  +  Smooth 

Dry  Depression 
Wadi 

Escarpments 
Salt  Marshes 
Salt  flats  +  Pipeline 
Flood  Plains 
Irrigated  fields 
Soil/sand/dirt  berms 
referred  to  as 
revetments 
Below  ground 
sand/dirt  trenches 
Unpaved  roads  + 
Ocean 

Loose  dirt  trails 
(less  than  1  road 
width)  +  Dry  Lake 
Dirt  and  concrete 
dikes  and  levees 


Output  is  the  shadow  map  from  8  different  angles  of  the 
same  terrain.  Print  and  align  the  maps  to  verify  terrain 
matches  in  all.  AOI  is  at  0,  0.  Final  image  contains 
artificial  features  used  for  quality  verification.  These 
image  contains  a  feature  of  known  height  in  known  terrain 
at  known  depression  angles.  Verify  shadow  length. 
Compare  with  test  #2  for  different  resolution  in  X  and  Y. 

Generate  airport  scene  and  verify  image  quality  of 
contained  features.  Print  out  feature  maps  and  align  to 
verify  constant  location  of  features.  Image  contains  all 
target  types  and  moving  targets. 

Print  the  gain  and  the  feature  maps  and  overlay  to  verify 
match  between  the  two. 

Run  only  the  first  few  images  and  compare  the  windy  palm 
trees  with  the  calm  palm  trees  in  test  4. 

Different  sizes  and  shapes  of  GRCA's  in  different  ARIES 
of  the  database.  Verify  features  and  terrain. 

Image  shows  the  coarse  texture  contained  within  the 
smooth  texture  background. 

Image  shows  the  fine  texture  contained  within  the  smooth 
texture  background. 

Image  shows  an  area  of  sand  dunes  within  the  smooth 
texture  background. 

Sample  image  of  feature  and  other  missing  items 
See  test  1 1 

Sample  image  of  feature 
Sample  image  of  feature 
Sample  image  of  feature 
Sample  image  of  feature 
See  test  1 1 
See  test  1 1 


See  test  1 1 

Sample  image  of  feature 
Sample  image  of  feature 


Also  includes  orchards,  power  plants,  telephone  poles  and 
wires  and  a  real  bridge! 
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23  Dirt  and  concrete 
walls  and  berms 

24  Quarries 

25  Divided  Hiway 

26  Area  Features 


27  Timing 

28  Odd  angles 

29  Linear 

30  Runways 

3 1  Orchards 


Sample  image  of  feature 

Sample  image  of  feature 
Sample  image  of  feature 

This  is  a  set  of  artificial  area  features  designed  to  challenge 
the  area  feature  routines.  Print  the  FID  maps  and  verily 
alignment. 

Run  a  set  of  images  on  sarsim  using  a  complete  but 
reduced  library  to  assess  actual  timing. 

General  test  at  odd  angles,  compare  FID  maps  with  those 
from  test  4. 

Test  pattern  of  linear  objects. 

Dual  runways 
Orchard  areas. 


5.3  Real  Time  Processing  Predictions 

Table  5.3-1  illustrates  the  allocated  and  projected  image  processing  times.  The  projected 
execution  at  NG  was  32.7  seconds.  The  actual  execution  time  for  the  same  image  was  24 
seconds.  Execution  time  will  vary  depending  on  image  contents.  During  build  3  delivery  at  NG 
reliable  execution  times  as  high  as  30  seconds  were  noted.  This  is  all  still  below  the  projected 
time. 

Table  5.3-1  Allocated  and  Projected  Image  Processing  Times 


Allocated 

Timing  Predictions 

Actual  Times 

CSC 

% 

Target 

Sun  Ultra  1 

Actual 

Predicted  vs 

Predicted 

% 

Host 

Timing 

Timings 

Actual 

Host 

Times 

Allocation 

on  Sun 

Timing 

Execution 

Required 

based  on 

Ultra  1 

Differences  Times  based 

Build  2 

on  Sun 

on  Build  2 

benchmark 

Ultra  1 

benchmark 

20.00 

22.14 

48.70 

90% 

44.00 

Process  RPSI  Data 

0.00% 

0.00 

0.00 

0.00 

0.00 

0.00% 

External  Obscuration 

5.00% 

1.00 

1.11 

1.11 

0.00 

0.00% 

Transform  AOI 

5.00% 

1.00 

1.11 

1.11 

0.00 

0.00% 

Generate  Point  Map 

25.00% 

5.00 

5.53 

8.00 

-2.47 

7.23 

22.11% 

3D  to  2D  Projection 

15.00% 

3.00 

3.32 

2.00 

1.32 

1.81 

5.53% 

Chip  &  Texture 

15.00% 

3.00 

3.32 

7.00 

-3.68 

6.32 

19.34% 

Final  Image  Proc. 

35.00% 

7.00 

7.75 

15.47 

-7.72 

13.98 

42.75% 

Image  Output 

0.00% 

0.00 

0.00 

3.72 

-3.72 

3.36 

10.28% 

Totals 

100.00% 

20.00 

22.14 

36.19 

-14.05 

32.70  100.00 

% 


Initialization 


300.00 


271.05 


Table  5.3-2  uses  the  measured  execution  times  from  the  SUN  scaled  to  the  original  image  size  of 
1024  x  512  pixels.  The  projected  execution  time  here  is  well  within  the  time  allotment  when 
processing  this  size  of  image. 

Table  5.3-2  Measured  Execution  Time  from  SAR 


Allocated 

Timing  Predictions 

Actual  Times 

CSC 

% 

Target 

Sun  Ultra  1 

Actual 

Predicted  vs 

Predicted 

% 

Host 

Timing 

Timings 

Actual 

Host 

Times 

Allocation 

on  Sun 

Timing 

Execution 

Required 

based  on 

Ultra  1 

Differences  Times  based 

Build  2 

on  Sun 

on  Build  2 

benchmark 

Ultra  1 

benchmark 

20.00 

22.14 

48.70 

90% 

44.00 

Process  RPSI  Data 

0.00% 

0.00 

0.00 

0.00 

0.00 

External  Obscuration 

5.00% 

1.00 

1.11 

1.11 

0.00 

Transform  AOI 

5.00% 

1.00 

1.11 

1.11 

0.00 

Generate  Point  Map 

25.00% 

5.00 

5.53 

2.48 

3.05 

2.24 

24.10% 

3D  to  2D  Projection 

15.00% 

3.00 

3.32 

0.62 

2.70 

0.56 

6.02% 

Chip  &  Texture 

15.00% 

3.00 

3.32 

2.17 

1.15 

1.96 

21.09% 

Final  Image  Proc. 

35.00% 

7.00 

7.75 

3.87 

3.88 

3.49 

37.58% 

Image  Output 

0.00% 

0.00 

0.00 

1.15 

-1.15 

1.04 

11.21% 

Totals 

100.00% 

20.00 

22.14 

10.29 

11.85 

9.30  100.00 
% 

Initialization 

300.00 

271.05 
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1.  INTRODUCTION 


Purpose/Scope  -  The  purpose  of  this  report  is  to  provide  the  test  results  obtained  from  the  JADS 
Phase  I  Acceptance  Test.  The  acceptance  test  for  the  Virtual  Surveillance  Target  Attack  Radar 
System  (VSTARS)  was  conducted  by  test  engineers  in  the  presence  of  government 
representatives  at  the  Northrop  Grumman  test  facility  located  at  Melbourne,  FL.  The  purpose  of 
the  test  was  to  operationally  test  VSTARS  using  two  Alpha  600  computers  and  Joint  STARS 
software  to  determine  if  all  contractual  requirements  were  satisfied. 

The  VSTARS  Acceptance  Test  for  JADS  Phase  I  demonstrated  the  ability  of  VSTARS  to  provide 
simulated  target  data  integrated  with  live  target  data  detected  by  Joint  STARS  via  the  Radar 
Processor  Simulator  and  Integrator  (RPSI).  The  simulated  target  data  was  detected  by  a  Moving 
Target  Indicator  (MTI)  Simulation  and  a  Synthetic  Aperture  Radar  (SAR)  simulation. 

Testing  was  conducted  on  two  Alpha  600  workstations  in  the  JADS  Laboratory  at  the  Northrop 
Grumman  ITF  using  the  JDS03_108B0  build.  The  VSTARS  ALPHA  600  workstations  and  the 
SGI  workstation,  the  PDU  Logger/Playback  equipment,  were  interconnected  on  the  Ethernet 
LAN”  shown  in  Figure  0-1  as  the  “Dl”  connection..  Seven  test  points  were  included  in  the 
procedure.  As  run  procedures  are  included  in  Appendix  A.  Five  test  points  were  successfully 
completed.  One  test  failed  two  steps  and  will  be  re-tested  during  Phase  II.  The  other  test  point 
failed  as  a  result  of  a  misinterpretation  between  Northrop  Grumman  and  JADS  Joint  Task  Force 
(JTF),  and  was  corrected  on  the  spot. 
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Preliminary  Performance  Assessment  for  Phase  I  of  VSTARS  Test  Report,  hereafter  referred  to 
as  the  Test  Report,  is  submitted  in  response  to  Contract  Data  Requirements  List  (CDRL)  item 
A006  under  the  Joint  Advanced  Distributed  Simulation  (JADS)  Program  Contract  No.  F30602- 
96-C-0281.  Content  and  format  are  in  accordance  with  the  requirements  of  ANSI  Z39.18 
Paragraph  5.1.1,  in  conformance  with  the  JADS  Statement  of  Work  (SOW). 


C2 


This  test  report  contains  the  results  from  the  12  November  1997  Preliminary  Assessment  Testing 
of  VSTARS.  The  following  test  points  were  executed: 

1 .  Programmable  Signal  Processor  (PSP)  Simulation 

2.  Navigation  Simulation 

3.  Radar  Simulation  and  Reporting  in  Real,  Mixed,  and  Virtual  Modes 

4.  Mixing  of  MTI  Radar  Virtual  Entities  with  Standard  E-8C  Graphics  Presentation 

5.  Real  and  Virtual  SAR  Reports 

6.  Normal  Joint  STARS  Operator  Functionality  While  Running  VSTARS 

7.  Ground  Interface  Unit  (GNIU)  Processing  Rate 

For  the  Preliminary  Assessment  Test,  the  VSTARS  default  values  were  input  according  to  Table 
0-1  below. 

Table  0-1  -  VSTARS  Default  Values 


1  -  Turn  off  PD  (Probability  of  Detection) 

F  (False) 

2  -  Turn  off  PFA  (Probability  of  False  Alarms) 

F  (False) 

3  -  Turn  off  CEP  (Circular  Error  of  Probability) 

F  (False) 

4  -  Turn  offCEPofX-RNG  and  D-RNG  Bias 

F  (False) 

5  -  Turn  off  Terrain  Screening 

F  (False) 

6  -  Display  Mixed  Mode  in  Same  Area 

F  (False) 

7  -  Display  Dummy  Targets 

F  (False) 

8  -  Minimum  Detectable  Velocity  (M/S) 

Set  to  0  for 
scenario 

9  -  Minimum  CEP  Cross  Range  Error 

Classified 

10  -  Minimum  CEP  Down  Range  Error 

Classified 

11  -  Minimum  PFA 

Classified 

12  -  Minimum  PD  Cross  Loss  (dB) 

Classified 

13  -  Monitoring  Virtual  Data  ID  (-1  for  all) 

-1 
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3.  TEST  RESULTS  SUMMARY 


The  results  of  the  Preliminary  Testing  of  VSTARS  is  illustrated  in  Table  0-2.  The  table  lists  the 
seven  test  points  evaluated  along  with  their  sub-categories. 

Table  0-2.  -  Test  Results  Summary 


TEST  POINT 

PASSED 

FAILED 

CORRECTIVE 

ACTION 

STR 

NUMBER 

1.  Programmable  Signal  Processor 
(PSP)  Simulation 

X 

None 

N/A 

•  “Live”  MTI  and  false  alarms  are 
displayed  in  the  GRCA. 

X 

None 

N/A 

•  Activity  is  shown  in  the  PSP 
fields  of  the  PKXOCO  window. 

X 

None 

N/A 

2.  Navigation  Simulation 

X 

None 

N/A 

•  Verify  position  updates  in  the 
System  Information  window. 

X 

None 

N/A 

•  Ground  truth  is  updating  in  the 
Navigation  Monitor  Sensors 
window. 

X 

None 

N/A 

•  Navigation  Status  updates  in  the 
Navigation  Monitor  of 

PKXOCO. 

X 

None 

N/A 

3.  Radar  Simulation  and  Reporting 
in  Real,  Mixed,  and  Virtual 

Modes 

X 

None 

N/A 

•  VSTARS  displays  “live”  MTI 
radar  on  the  Graphics  Display 
(GD). 

X 

None 

N/A 

•  VSTARS  displays  only  virtual 
MTI  radar  on  the  Graphics 
Display 

X 

None 

N/A 

•  VSTARS  displays  virtual  and 
“live”  MTI  radar  in  both  the  live 
and  virtual  ACs  and  the  GRCA. 

X 

None 

N/A 

4.  Mixing  of  MTI  Radar  Virtual 
Entities  with  Standard  E-8C 
Graphics  Presentation 

X 

None 

N/A 
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TEST  POINT 

PASSED 

FAILED 

CORRECTIVE 

ACTION 

STR 

NUMBER 

•  Graphic  Features  are  displayed 
at  every  expansion. 

X 

None 

N/A 

•  All  available  Geographical  Data 
is  displayed  on  the  Graphic 
Display. 

X 

None 

N/A 

•  SARs  available  in  the  SAR  File 
Select  are  shown  on  the  GD. 

X 

None 

N/A 

5.  Real  and  Virtual  SAR  Reports 

X 

Fixed  on  the  spot 
via  a  software 
patch. 

#30484 

•  Live  SARs  are  displayed  on  the 
GD  at  proper  orientation  from 
the  aircraft  with  acceptable 
fidelity.  Because  this  is  in  the 
lab,  live  SARs  are  just  noise. 

X 

None 

N/A 

•  Virtual  SARs  are  displayed  on 
the  GD  at  proper  orientation 
from  the  aircraft  with  acceptable 
fidelity  using  the  SAR 

Simulation. 

X 

None 

N/A 

•  Mixed  SARs  are  displayed  on 
the  GD  at  proper  orientation 
from  the  aircraft  with  acceptable 
fidelity  using  the  SAR 

Simulation. 

X 

Live  SARs  were 
displayed  instead 
of  virtual.  This 
was  changed  on 
the  spot  so  that 
mixed  SARs  are 
displayed  using 
the  SAR 
Simulation. 

#30484 

6.  Normal  Joint  STARS  Operator 
Functionality  While  Running 
VSTARS 

X 

STRs  written  for 
two  functions 
that  did  not 
work. 

#30485 

#30486 

•  Route  functionality  is  available 
while  running  VSTARS. 

X 

None 

N/A 

•  E-Track  functionality  is  available 
while  running  VSTARS. 

X 

None 

N/A 

•  A-Track  functionality  is  available 
while  running  VSTARS. 

X 

None 

N/A 
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TEST  POINT 

PASSED 

FAILED 

CORRECTIVE 

ACTION 

STR 

NUMBER 

•  History  Playback  functionality  is 
available  while  running 

VSTARS. 

X 

None 

N/A 

•  Order  of  Battle  functionality  is 
available  while  running 

VSTARS. 

X 

None 

N/A 

•  Engagement  Point  functionality 
is  available  while  running 
VSTARS. 

X 

None 

N/A 

•  Radar  Screening  functionality  is 
available  while  running 

VSTARS. 

X 

None 

N/A 

•  User  Defined  Activity 

functionality  is  available  while 
running  VSTARS. 

X 

None 

N/A 

•  Timeline  Impact  functionality  is 
available  while  running 

VSTARS. 

X 

Prevent  PFZTLP 
process  from 
crashing  when 
performing 
Timeline  Impact 
in  the  Flight  Path 
Mode. 

#30485 

•  Update  Radar  Status 

functionality  is  available  while 
running  VSTARS. 

X 

None 

N/A 

•  Jammer  Sector  functionality  is 
available  while  running 

VSTARS. 

X 

None 

N/A 

•  Sector  and  Area  Blanking 
functionality  is  available  while 
running  VSTARS. 

X 

None 

N/A 

•  Bearing  and  Range  functionality 
is  available  while  running 
VSTARS. 

X 

None 

N/A 

•  Locate  Entity  functionality  is 
available  while  running 

VSTARS. 

X 

None 

N/A 
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TEST  POINT 

PASSED 

FAILED 

CORRECTIVE 

ACTION 

STR 

NUMBER 

•  Point  functionality  is  available 
while  running  VSTARS. 

X 

Prevent  the 
PTMWND 
process  from 
crashing  the 
windows  handler 
when  the  Point 
function  is 
activated. 

#30486 

•  Cursor  Coordinate  functionality  is 
available  while  running  VSTARS. 

X 

None 

N/A 

7.  Ground  Interface  Unit  (GNIU) 
Processing  Rate 

X 

Request  by 
JADS  JTF  to 
change  DIS 
Monitor  format. 

#30487 

•  With  recording  turned  off,  and  a  rate 
of  300  selected,  PDUs  are  played 
back  at  a  rate  of  300  PDUs  per 
second. 

X 

None 

N/A 

•  With  recording  turned  off,  and  a  rate 
of  500  selected,  PDUs  are  played 
back  at  a  rate  over  350  PDUs  per 
second. 

X 

None 

N/A 

•  With  enhanced  recording  turned  on, 
PDUs  are  played  back  at  a  rate  over 
350  PDUs  per  second. 

X 

None 

N/A 

•  With  recording  turned  off,  and  a  rate 
of  768  selected  PDUs  are  played 
back  at  a  rate  over  350  PDUs  per 
second. 

X 

None 

N/A 
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4. 


TEST  RESULTS 


4.1  Programmable  Signal  Processor  (PSP)  Simulation 

The  PSP  Simulator  will  receive  setup  and  parameter  data  from  the  Radar  Data  Processor  (RDP). 
It  will  generate  node  1  data  (target  data)  messages  and  send  them  to  the  RDP  for  target 
association. 

JADS  processes  including  Navigation  Simulation,  MTI  Simulation,  SAR  Simulation,  and 
Operations  and  Control  Operation  Task  were  initialized  and  the  system  was  put  into  the  operate 
mode.  Distributed  Interface  Simulation  (DIS)  was  not  activated  so  that  only  PSP  generated 
targets  were  displayed.  A  Ground  Reference  Coverage  Area  (GRCA)  was  created  and  approved. 
“Live”  MTI  radar  was  displayed  in  the  GRCA  indicating  active  PSPs.  In  addition,  PSP  activity 
was  verified  in  the  PKXOCO  window  under  the  PSP  1 ,  2,  and  3  activity  fields. 

4.2  Navigation  Simulation 

The  Navigation  Simulation  will  simulate  the  position,  altitude,  and  speed  of  an  E-8C  aircraft  flying 
a  specified  orbit.  It  also  will  provide  this  data  to  the  navigation  software  associated  with  the  radar 
function.  Additionally,  it  will  provide  required  data,  reports,  and  messages  to  the  radar  subsystem 
as  required  by  the  RDP. 

To  verify  Navigation  Simulation  operation,  the  E-8C  System  Information  window  was  observed 
for  updates  to  platform  position.  Also,  in  the  Simulation  Navigation  window,  the  Sensor's 
Monitor  was  displayed  to  show  ground  truth  updates.  Finally,  in  PKXOCO  process  window,  the 
Nav  Monitor  was  accessed  to  confirm  that  nav  status  was  updating.  The  Navigation  Simulation 
sent  required  updates  to  the  navigation  software. 

4.3  Radar  Simulation  and  Reporting  in  Real,  Mixed,  and  Virtual  Modes 

VSTARS  MTI  simulation  will  operate  in  three  modes:  only  live  targets  displayed,  mixed  (live  and 
virtual)  displayed,  and  virtual  targets  only  displayed.  The  difference  between  live  and  virtual 
targets  will  not  be  visually  distinguishable. 

The  first  set  of  procedures  under  this  test  were  used  to  verify  the  ability  of  VSTARS  to  operate  in 
the  live  only  mode.  Prior  to  testing,  two  areas  were  created  to  display  live  targets.  One  was  a 
live  only  area  and  the  other  was  mixed,  live  and  virtual.  Distributed  Interactive  Simulation 
(DISSIM)  and  SGI  (Silicon  Graphics  Indy)  play  back  were  not  initialized.  False  alarms  were 
turned  off  and  PSP  load  was  set  to  90%  in  order  to  make  live  areas  more  visible.  History 
playback  was  turned  on  and  set  in  the  integrate  mode.  In  both  of  the  areas,  live  targets  were 
clearly  visible. 

The  next  step  was  to  operate  in  the  virtual  only  mode.  PSP  activity  was  set  to  0  to  stop  “live” 
processing.  False  Alarms  were  then  turned  off  to  make  it  easier  to  see  virtual  targets.  The  DIS 
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Monitor  and  Interface  Unit  was  activated  to  enable  the  processing  of  Protocol  Data  Units  (PDUs) 
across  the  Large  Area  Network  (LAN).  A  TELNET  connection  was  made  to  the  SGI,  and  the 
JADS  Player  was  started  to  replay  a  stored  scenario  file.  Finally,  an  Attack  Control  (AC)  area 
was  initiated  over  the  virtual  area  to  increase  the  update  rate  and  probability  of  detection.  Only 
virtual  targets  were  displayed  on  the  Graphics  Display  (GD). 

The  final  procedure  demonstrated  that  both  mixed  and  virtual  targets  could  be  shown  at  the  same 
time  on  the  GD.  With  the  SGI  scenario  running,  selection  was  made  to  mix  live  and  virtual  data. 
The  PSP  load  was  reset  to  the  default  of  15%  capacity  and  false  alarms  were  turned  back  on.  The 
virtual  targets  from  the  scenario  continued  to  display  along  with  the  targets  in  the  live  and  mixed 
areas. 


4.4  Mixing  of  MTI  Radar  Virtual  Entities  with  Standard  E-8C  Graphics 
Presentation 

The  VSTARS  will  permit  the  mixing  of  MTI  radar  virtual  entities,  terrain,  and  SAR  images 
seamlessly  with  the  actual  radar  images  produced  by  the  E-8C. 

Live  “noise”  targets  and  virtual  targets  replayed  via  the  Silicon  Graphics  machine  were  displayed 
on  the  Graphics  Display.  While  running  in  the  mixed  mode,  Graphics  Features  and  Graphics 
Underlays  were  displayed  successfully  at  all  scale  expansions.  Following  the  display  of  standard 
graphical  features,  the  SAR  File  Selection  List  was  accessed  to  select  previously  recorded  SAR 
files  for  exhibition  on  the  Operator  Workstation.  Standard  E-8C  Graphics  and  SARs  were 
successfully  integrated  with  mixed  MTI  and  virtual  entities. 

4.5  Real  and  Virtual  SAR  Reports 

VSTARS  will  permit  the  SAR  report  to  consist  of  either  a  live  report  or  a  virtual  report.  If  the 
center  position  of  the  SAR  falls  within  a  virtual  area,  the  SAR  will  be  virtual.  If  it  falls  within  a 
live  area,  the  SAR  will  be  live,  and  if  it  falls  within  a  mixed  area,  the  SAR  will  be  virtual. 

Virtual  SARs  were  tested  first.  At  each  portion  of  the  E-8C  orbit:  Eastern,  Central,  and 
Western,  two  virtual  SARs  were  requested  and  approved.  All  six  of  these  SARs  were  displayed 
on  the  graphics  display  at  proper  orientation  from  the  aircraft  and  at  acceptable  fidelity. 

The  next  SAR  requested  was  in  the  location  of  fixed  virtual  targets.  Upon  display  of  the  SAR, 
simulated  cartographic  data  was  shown  along  with  bright  spots  representing  the  fixed  targets  as 
expected. 

Live  SARs  were  then  tested,  within  the  live  only  area.  At  each  portion  of  the  E-8C  orbit:  Eastern, 
Central,  and  Western,  two  live  SARs  were  requested  and  approved.  The  live  SARs  were 
displayed  at  proper  orientation  as  noise  because  the  test  was  run  in  the  laboratory  setting  without 
a  radar  sensor. 
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The  purpose  of  the  final  step  was  to  test  SARs  in  a  mixed  area.  At  each  portion  of  the  E-8C 
orbit:  Eastern,  Central,  and  Western,  two  mixed  SARs  were  requested  and  approved.  The  SARs 
in  the  mixed  areas  were  displayed  as  live  SARs  as  designed  by  Northrop  Grumman.  However,  at 
this  point  it  was  stated  that  JADS  JTF  required  SARs  in  mixed  areas  to  be  virtual  instead  of  live. 
A  software  patch  was  installed  to  temporarily  correct  the  problem.  A  Software  Trouble  Report 
(STR)  #30484,  was  written  to  enable  virtual  only  SARs  in  mixed  areas.  This  functionality  will  be 
re-tested  in  Phase  II. 

4.6  Normal  Joint  STARS  Operator  Functionality  While  Running  VSTARS 

In  all  modes  of  operation,  VSTARS  will  permit  all  the  installed  operator  workstation  software  to 
function  without  the  occurrence  of  abnormal  fault  messages  caused  by  VSTARS. 

Various  Joint  STARS  operator  functions  were  examined  to  verify  that  VSTARS  did  not  interfere 
with  their  operation.  The  route  function  was  the  first  application  tested.  Both  cartographic  and 
arbitrary  routes  were  initialized  and  modified.  The  cartographic  route  was  deleted.  Route 
functionality  is  available  while  running  VSTARS. 

Tracking  functionality  was  then  demonstrated.  A  constrained  extrapolated  track  (E-Track)  was 
initiated  using  the  “radar  derived”  option  and  the  “segment  method”  to  determine  the  target 
centroid.  After  successful  E-Track  initiation,  an  unconstrained  Automated  Track  (A-Track)  was 
initiated  using  “specified  speed  and  course”  and  the  “center  method”  to  determine  the  target 
centroid.  A  second  A-Track  was  then  initiated  using  the  “differential  position”  option  and  the 
“box  method”  to  determine  the  target  centroid.  The  second  A-Track  was  selected  for  automatic 
repositioning  with  its  associated  Attack  Control  (AC)  area.  Both  E-Track  and  A-Track 
functionality  were  available  while  running  VSTARS. 

The  next  function  verified  was  history  playback.  History  was  selected  in  the  integration  mode,  by 
frame,  and  set  to  start  10  minutes  prior  to  current  system  time.  MTI  history  was  displayed 
continuously.  At  this  point,  history  was  paused  and  sliding  window  was  selected  as  well  as 
integration.  The  window  size  and  advance  were  set  to  10  and  history  replay  was  restarted. 
History  playback  showed  only  the  selected  window  of  the  integration  as  expected.  Then,  history 
was  paused  again  and  both  integrate  and  sliding  window  were  deselected.  When  restarted,  history 
played  back  in  the  non-integrated  mode.  History  Playback  functionality  is  available  while  running 
VSTARS. 

Order  of  Battle  capability  was  demonstrated.  Order  of  Battle  points,  lines,  and  areas  were  all 
created,  modified,  and  deleted.  Normal  Order  of  Battle  functionality  is  available  while  running 
VSTARS. 

Engagement  Point  (EP)  operability  was  confirmed.  First,  a  base  engagement  point  was  created. 
In  order  to  pair  the  engagement  point,  an  E-Track  was  initiated  15  km  in  front  of  the  base 
engagement  point.  The  existing  EP  was  associated  to  the  E-Track.  A  red  EP  was  shown  on  the 
GD,  and  time  of  arrival  information  was  displayed  in  the  Time  of  Arrival  (TOA)  data  field  as 
usual.  Engagement  Point  functionality  is  available  while  running  VSTARS. 


The  next  capability  demonstrated  was  Radar  Screening.  With  the  SGI  scenario  replay  continuing, 
an  A-Track  was  initiated  on  virtual  targets.  The  Radar  Screening  was  accessed  and  the  fixed 
ground  point  option  was  selected.  Upon  depressing  the  ENTER  key,  a  magenta  line  was 
displayed  on  the  flight  path  indicating  the  target  was  visible.  As  with  typical  Joint  STARS 
missions,  the  turn  was  shown  as  cyan  to  indicate  the  target  would  not  be  visible.  For  the  next 
step,  the  route  segment  option  was  chosen.  Upon  depressing  the  ENTER  key,  the  selected  route 
segment  was  colored  magenta  indicating  that  the  route  segment  was  not  screened.  Route 
Screening  functionality  is  available  while  running  VSTARS. 

User  Defined  Activity  Areas  (UDA)  were  verified  next.  A  UDA  with  a  threshold  of  5  was  built  5 
kilometers  in  front  of  an  virtual  target  group.  The  4X4  UDA  appeared  on  the  GD.  When  more 
than  5  targets  entered  the  UDA,  an  alert  “UDA  THRESHOLD  EXCEEDED”  was  displayed,  and 
the  UDA  border  thickened.  User  Defined  Activity  functionality  is  available  while  running 
VSTARS. 

The  next  step  was  to  confirm  the  functionality  of  Timeline  Impact.  The  Timeline  Impact  Tabular 
Display  (TL  IMPACT  TD)  was  selected  from  the  Process  RSR  push-button  menu.  The  Flight 
Path  (FP)  option  was  selected.  Projected  time  for  the  simulated  E-8C  was  input,  and  the  start 
time  set  for  two  minutes  later.  The  chosen  duration  was  60  minutes  and  all  RSRs  were  selected 
for  impact  analysis.  Upon  depressing  the  ENTER  key,  this  software  application  timed  out  and 
died.  The  process  was  restarted,  but  again,  the  application  timed  out  and  died.  The  test  was  run 
again  after  the  Assessment  Test  without  any  of  the  VSTARS  processes  running.  Again,  this 
process  did  not  work.  An  STR  (#30485)  has  been  written  to  correct  this  problem.  This  test  will 
be  re-run  in  Phase  II. 

Update  Radar  Status  was  the  next  function  to  verify.  The  Approved  RSR  List  TD  was  displayed 
on  the  GD  for  monitoring.  The  Update  Radar  Status  button  was  depressed,  and  immediately,  the 
Approve  List  was  updated.  The  Update  Radar  Status  functionality  is  available  while  running 
VSTARS. 

Jammer  Sector  was  tested  next.  The  Jammer  Sector  TD  was  selected  from  the  Radar 
Management  push-button  menu.  A  jammer  sector  was  displayed  on  the  GD  as  a  wedge  centered 
at  the  indicated  position.  Jammer  Sector  functionality  is  available  while  running  VSTARS. 

The  following  steps  confirmed  the  operation  of  Sector  Blanking  and  Area  Blanking.  First,  a 
Sector  Blanking  Service  Request  was  initiated  over  SGI  play  back  targets  using  a  fixed  ground 
point,  azimuth  width  of  5,  and  duration  of  0.  After  approval  of  the  blanked  sector,  no  targets 
were  displayed  in  the  sector.  Then,  an  Area  Blanking  Service  Request  was  initialized.  Again,  it 
was  centered  over  virtual  targets.  Upon  approval,  a  10  X  10  blanked  area  was  displayed  on  the 
GD.  There  were  no  target  detections  in  the  area.  The  Sector  and  Area  Blanking  functionality  are 
available  while  running  VSTARS. 

The  last  step  in  this  test  was  to  ensure  that  the  functions  available  from  the  pull-down  menu  were 
available  with  VSTARS  running.  Bearing  and  range  and  free  form  were  the  first  applications 


C14 


tested.  Both  worked  as  expected.  Next,  locate  entity  was  selected  using  an  RSR  number  for 
input.  A  large  arrow  appeared  on  the  GD  identifying  the  location  of  the  requested  RSR.  The 
next  application  was  the  Point  function.  Upon  selection  of  this  option,  window  handler  crashed 
and  restarted.  This  application  is  not  available  on  the  Alpha  600  at  this  time.  An  STR  (#30486) 
has  been  written  and  will  be  re-run  in  Phase  II.  Cursor  Coordinate  functionality  was  tested  next. 
A  position  was  input.  When  the  ENTER  key  was  depressed,  the  position  was  converted  into 
available  coordinate  systems  within  the  TD.  All  pull-down  menu  functions  were  available  while 
running  VSTARS  except  the  point  function. 

4.7  Ground  Interface  Unit  (GNIU)  Processing  Rate 

The  Ground  Interface  Unit  (GNIU)  receives  and  processes  DIS  2.0.4  Entity  State  Protocol  Data 
Units  (ESPDUs)  at  a  rate  greater  than  350  ESPDUs  per  second. 

The  JADS  Player  on  the  SGI  was  used  to  replay  protocol  data  (PDUs)  units  at  various  rates  to 
show  the  GNIU  can  exceed  350  ESPDUs  per  second.  At  the  various  rates  recording  was  turned 
off,  set  to  a  simplified  recording  level  of  1,  or  the  enhanced  recording  level  of  2.  The  PDU  rate 
was  monitored  via  the  DIS  Monitor  window.  This  window  is  displayed  in  hex  and  updates  every 
two  seconds.  As  requested  by  JADS  JTF,  an  STR  (#30487)  was  written  to  label  DIS  Monitor 
columns  and  convert  hex  numbers  to  decimal. 

Table  0-3  displays  the  results  of  the  this  test.  The  first  replay  rate  used  was  a  rate  of  300  with 
recording  turned  off.  The  Average  number  of  PDUs  was  300.  Next,  the  rate  was  set  to  500. 
With  recording  turned  off,  the  number  of  PDUs  replayed  was  500;  with  recording  set  to  1,  the 
replay  rate  averaged  455,  and  with  level  2,  the  rate  averaged  450.  The  final  recording  rate 
selected  was  768  with  recording  turned  off.  The  PDU  rate  per  second  averaged  750.  This  test 
verified  that  the  GNIU  processing  rate  exceeds  350  PDUs  per  second. 


Table  0-3  -  GNIU  Processing  Rate 


r-300  recording  off 


r  9ff8 

40952 

598 

299 

9da2 

40354 

607 

303.5 

9b43 

39747 

591 

295.5 

98f4 

39156 

594 

297 

96a2 

38562 

38562 

19281 

r-500  recording  off 


1a62 

6754 

1003 

501.5 

1677 

5751 

996 

498 

1293 

4755 

999 

499.5 

eac 

3756 

1001 

500.5 

ac3 

2755 

999 

499.5 

6dc 

1756 

1000 

500 

2f4 

756 

756 

378 

r-500  recording  set  to  1 


1 1c6 

4550 

992 

496 

de6 

3558 

995 

497.5 

a03 

2563 

994 

497 

621 

1569 

994 

497 

23f 

575 

575 

287.5 

r-500  recording  set  to  2 


20d2 

8402 

900 

450  1 

1d4e 

7502 

896 

448 

19ce 

6606 

893 

446.5 

1651 

5713 

896 

448 

1 2d  1 

4817 

894 

447 

f53 

3923 

899 

449.5 

bdO 

3024 

893 

446.5 

853 

2131 

2131 

1065.5 

r-768  recording  off 


2e2c7 

189127 

1496 

748 

2dcef 

187631 

1498 

749 

2d715 

186133 

1499 

749.5 

2d13a 

184634 

1497 

748.5 

2cb61 

183137 

1503 

751.5 

2c582 

181634 

1503 

751.5 

2bfa3 

180131 

1506 

753 

2b9c1 

178625 

178625 

89312.5 

Average  PDU  Rate  Per  Second 


299  r-300  recording  off 


500  r-500  recording  off 


455  r-500  recording  1 


448  r-500  recording.^ 


750  r-768  recording  off 
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5.  PHASE  I  ASSESSMENT  TEST  SUMMARY 


Overall,  the  Phase  I  Assessment  Test  was  successful.  Out  of  over  200  procedural  steps,  only 
three  failed.  Of  these  three,  the  mixed  area  SARs  have  already  been  corrected,  and  Timeline 
Impact  and  the  Point  function  are  considered  to  be  minor  problems.  In  addition  to  test  findings, 
JAD  JTF  also  requested  a  more  user  friendly  DIS  Monitor.  For  example,  the  columns  should  be 
labeled  and  the  numbers  in  decimal  format  versus  hex  format.  After  the  Software  Trouble 
Reports  (#30484,  30485,  30486,  30487)  are  incorporated,  the  three  failed  areas  mentioned  above 
will  be  re-run  during  Phase  II  testing. 
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Appendix  D 

Characterization  Reports  From  JADS  JTF/ETE  (N&E) 

MEMORANDUM  FOR  RECORD  16  Dec  97 


FROM  :  JADS  JTF/ETE(N&E) 

SUBJECT:  Characterization  Report 

1.  Event  Name:  TCAC-Grumman  T-l  Characterization. 

2.  Date/Time:  1  Dec  97;  1100  -  1500. 

3.  Site(s):  Grumman  -  Melbourne,  FL;  JADS  TCAC  -  Albuquerque,  NM. 

4.  Description:  The  characterization  involved  base-lining  the  performance  of  the  T-l  link 
between  Grumman  and  the  TCAC.  A  variety  of  Networking  and  Engineering  tests  were  used  to 
calculate  the  round  trip  latencies  and  determine  the  maximum  rate  at  which  DIS  PDUs  could  be 
transmitted  before  drop-outs  occurred.  The  tests  used  to  gather  this  data  included  the  No  Load 
Latency  Test,  the  Loaded  Latency  Test,  PDU  Verification  Test  and  the  Stress  Test.  Additionally, 
Spectrum  recorded  the  bandwidth  utilized  across  the  T-l  link. 

5.  Objective:  Assess  (characterize)  the  TCAC-Grumman  T-l  Link  with  and  without  network 
analysis  tools  operating. 

5.1  Determine  the  “no  load”  latency  using  pings. 

5.2  Determine  the  loaded  latency  using  pings  at  varying  rates. 

5.3  Successful  transmission  of  DIS  PDUs  across  the  link. 

5.4  Determine  the  packet  rate  (packets  per  second)  in  which  DIS  PDUs  are  lost. 

5.5  Determine  the  impact  of  Spectrum  on  latencies  and  data  rates. 

6.  Results: 

6.1.  Data  Recorded  (with  Spectrum  operational) 

6.1.1  No  Load  Latency  Test:  The  first  test  conducted  was  the  no  load  latency  test  using 
pings.  This  test  involved  using  32  pings  for  measuring  the  baseline  round  trip  latency  from  the 
TCAC  to  Grumman.  The  minimum  load  for  these  pings  were  64  bytes.  This  test  yielded  the 
following  results: 


D  4 


Table  6.1.1.  Latency  Test  -  No  Load 


Date 

Source 

Destination 

No.  Pkts 

Round-trip  Time  (ms) 

Minimum 

Maximum 

Average 

1  Dec  97 

TCACJndy 

Grumman_Indy 

32 

57 

71 

57 

6. 1 .2.  Loaded  Latency  Test:  The  second  test  consisted  of  transmitting  loaded  pings, 
144  bytes,  at  several  different  rates  and  packet  sizes  to  determine  the  round  trip  latencies-  see 
table  6.1.2. 


Table  6.1.2.  Latency  Test  -  Loaded 


Date 

Source 

Destination 

Pkt 

Rate(sec) 

Pkts 

Sent/Rec 

Round 

-trip  Time  (ms) 

Min 

Max 

Ave 

1  Dec  97 

TCACJndy 

Grumman Jndy 

.01 

320/320 

58 

163 

.005 

640/639 

58 

61 

.0025 

1280/1279 

58 

124 

58  j 

.00125 

2560/2559 

58 

167 

58  | 

6.1.3.  PDU  Verification  Test:  The  next  test  used  PDUs  generated  at  100  packets  per 
second  from  JANUS  and  recorded  by  the  JADS  logger.  The  PDUs  were  replayed  from  the 
TCAC  and  logged  at  Grumman  -  see  table  6.1.3. 


Table  6.1.3.  PDU 

Verification  Test 

Date 

Location 

No.  of  PDUs 
Received 

No.  of  PDUs 
Transmitted 

No.  of  PDUs 
Out-of-order 

No.  of  PDUs 
>  1  sec. 

5  Nov  97 

Grumman 

11859 

11859 

0 

0 

6.1.4.  Stress  Test:  This  test  involved  replaying  the  ESPDU  file  at  its  recorded  excepted 
data  rate  (EDR),  then  increasing  the  EDR  by  100  packets  per  second  incrementally  to  five  times 
the  EDR.  During  the  actual  playback  of  the  PDU  log  file,  pings  were  transmitted  and  latency 
statistics  recorded.  Spectrum  also  recorded  the  bandwidth  utilized. 


Table  6.1.4.  Stress  Test 


Rate 

(PDU/sec) 

Ping  Time  (ms) 
Min/Max/Ave 

Number  PDUs 
Received 

Bandwidth 

Util.  (%) 

1  X  E.R. 

60/120/61 

11859 

10 

59/90/68 

11858 

21 

3  X  E.R. 

300 

59/90/64 

11853 

28 

4  X  E.R. 

400 

57/210/79 

11847 

5  X  E.R. 

500 

57/840/256 

45  1 
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6.2.  Data  Recorded  (without  Spectrum  operational) 

6.2. 1  No  Load  Latency  Test:  The  first  test  conducted  was  the  no  load  latency  test  using 
pings.  This  test  involved  using  32  pings  for  measuring  the  baseline  round  trip  latency  from  the 
TCAC  to  Grumman.  The  minimum  load  for  these  pings  were  64  bytes.  This  test  yielded  the 
following  results: 


Table  6.2. 1 .  Latency  Test  -  No  Load 


Date 

Source 

Destination 

No.  Pkts 

Round-trip  Time 

(ms) 

Minimum 

Maximum 

Average 

dhhh 

Grummanjndy 

32 

57 

76 

57 

■  ■ 

6.2.2.  Loaded  Latency  Test:  The  second  test  consisted  of  transmitting  loaded  pings, 
144  bytes,  at  several  different  rates  and  packet  sizes  to  determine  the  round  trip  latencies-  see 
table  6.2.2. 


Table  6.2.2.  Latency  Test  -  Loaded 


Pkt 

Rate(sec) 

Pkts 

Sent/Rec 

Round-trip  Time  (ms) 

Min 

H®I 

ES8SSS818EM 

.01 

320/320 

58 

.005 

640/639 

58 

64 

58 

.0025 

1280/1279 

58 

101 

58 

.00125 

2560/2559 

58 

161 

58 

6.2.3.  PDU  Verification  Test:  The  next  test  used  PDUs  generated  at  100  packets  per 
second  from  JANUS  and  recorded  by  the  JADS  logger.  The  PDUs  were  replayed  from  the 
TCAC  and  logged  at  Grumman  -  see  table  6.2.3. 


Table  6.2.3.  PDU  Verification  Test 


Date 

Location 

No.  of  PDUs 
Received 

No.  of  PDUs 
Transmitted 

No.  of  PDUs 
Out-of-order 

No.  of  PDUs 
>  1  sec. 

5  Nov  97 

Grumman 

11859 

11859 

0 

0 

6.2.4.  Stress  Test:  The  last  test  conducted  was  the  Stress  Test.  The  test  involved 
replaying  the  ESPDU  file  at  its  recorded  EDR,  then  increasing  the  EDR  by  100  packets  per 
second  incrementally  to  five  times  the  EDR.  During  the  actual  playback  of  the  PDU  log  file, 
pings  were  transmitted  and  latency  statistics  recorded;  however,  bandwidth  data  was  not 
collected. 
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Table  6.2.4.  Stress  Test 


Rate 

(PDU/sec) 

Ping  Time  (ms) 
Min/Max/Ave 

Number  PDUs 
Received 

Bandwidth 

Util.  (%) 

1  X  E.R. 

100 

60/70/60 

11859 

2  X  E.R. 

200 

58/92/62 

11858 

3  X  E.R. 

300 

59/89/63 

11853 

4  X  E.R. 

400 

57/120/71 

11848 

5  X  E.R. 

500 

57/600/216 

10759 

6.3.  Problems: 

6.3.1.  The  characterization  process  was  delayed  for  a  couple  of  hours  due  to  the 
monthly  crypto  reloading. 

6.3.2.  During  the  PDU  Verification  test  additional  DIS  PDUs  were  being  logged  at  the 
Grumman_Indy  computer.  The  DIS  PDUs  logged  at  Grumman  exceeded  the  number  contained 
in  the  replayed  log  file  sent  from  the  TCAC.  The  Grumman_Indy  computer  can  receive  data  from 
both  Grumman  and  the  TCAC.  The  test  coordinator  contacted  Grumman  and  inquired  as  to 
whether  they  had  a  system  broadcasting  to  the  Grumman_Indy  computer  on  port  3000. 
Grumman’s  initial  response  to  our  question  was  no.  Upon  our  investigation,  which  consisted  of 
viewing  traffic  along  port  3000,  we  observed  the  Grumman_Indy  computer  receiving  data  long 
after  we  had  stopped  broadcasting.  Apparently,  Grumman  had  concluded  a  test  last  week,  but 
their  Alpha  computer  was  still  sending  data  to  port  3000.  Finally,  Grumman  disconnected  their 
Alpha  computer  and  we  resumed  testing  with  no  further  problems  or  delays. 

6.3.3.  The  IDNX  router  card  throughput  is  presently  less  than  the  vendor’s  specification 
and  the  allocated  capacity  of  the  T-l  link.  The  performance  can  be  improved  if  we  used  bridging 
as  opposed  to  Internet  Protocol  (IP)  routing  of  DIS  PDUs.  This  procedure  was  assessed  using 
the  JADS  testbed.  The  effect  of  bridging  doubled  the  reliable  transmitted  data  rate  to  800  packets 
per  second.  However,  this  configuration  does  not  support  the  current  network  design  of  the  ETE 
test.  The  vendor  was  contacted  about  the  shortcoming,  and  they  are  pursuing  every  avenue  to 
determine  if  the  performance  problem  is  their  software,  Cisco's  software,  or  hardware  related. 

6.3.4  Although  the  utilized  bandwidth  was  accurately  recorded  using  Spectrum  for  one, 
two,  and  three  times  the  EDR,  the  values  for  four  and  five  times  seemed  lower  than  expected. 

The  percentages  were  20  and  19,  respectively.  The  data  collection  sampling  interval  was  initial 
set  for  every  30  seconds.  Consequently,  the  DIS  PDUs  were  not  replayed  within  the  time 
window  and  the  results  were  not  representative  of  the  EDR.  The  settings  were  reduced  to  10 
second  intervals  and  Spectrum’s  packet  rate  data  was  monitored  to  ensure  the  values  reflected  the 
EDR. 

6.3.5.  The  maximum  value  recorded  for  each  set  of  pings  represented  an  outlier  of  the 
data  set.  This  maximum  value  far  exceeded  the  values  recorded  and  consistently  appeared  at  the 
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beginning  of  the  recorded  data  set.  Since  this  anomaly  happened  infrequently  during  a  ping  data 
set,  the  results  were  not  statistically  significant.  N&E  will  continue  to  investigate  this  issue. 

6.4.  Summary:  At  lOOOhrs  the  characterization  team  began  the  preliminary  checks  of  the 
network.  About  one  hour  later,  the  characterization  began  and  each  one  of  the  tests  was  executed 
in  sequence.  The  test  results  were  stable  and  fairly  predicable.  During  the  Stress  Test  several 
DIS  PDUs  were  lost  at  three  and  four  times  the  EDR  but  did  not  achieve  a  notable  percentage. 
However,  at  five  times  the  EDR  several  hundred  DIS  PDUs  were  dropped— approximately  nine 
percent.  Currently,  the  highest  EDR  transmissions  across  this  link  should  not  exceed  400  packets 
per  second;  therefore,  the  link  can  successfully  handle  the  expected  traffic.  The  characterization 
test  also  involved  capturing  the  impact  of  the  network  analysis  tool.  The  results  indicated  that 
Spectrum  minimally  increased  the  amount  of  network  traffic  and  added  an  insignificant  amount  of 
latency. 


GREGORY  P.  DEWIT 
CPT(P),  FA 
Test  Analyst 
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MEMORANDUM  FOR  RECORD 


26  Feb  98 


FROM  :  JADS  JTF/ETE/N&E 
SUBJECT:  ETE  Characterization  Report 

1.  Event  Name:  White  Sands  Missile  Range  (WSMR)/TCAC-A  Annex  T-l  Characterization. 

2.  Date/Time:  3  Feb  98;  1100  -  1500. 

3.  Sites:  WSMR,  NM;  JADS  TCAC  -  Albuquerque,  NM. 

4.  Description:  The  characterization  involved  base-lining  the  performance  of  the  T-l  link 
between  WSMR  and  the  TCAC-A.  A  variety  of  Networking  and  Engineering  tests  were  used  to 
calculate  the  round  trip  latencies  and  determine  the  maximum  rate  at  which  DIS  PDUs  could  be 
transmitted  before  drop-outs  occurred.  The  tests  used  to  gather  this  data  included  the  No  Load 
Latency  Test,  the  Loaded  Latency  Test,  PDU  Verification  Test,  and  the  Stress  Test. 

5.  Objective:  Assess  (characterize)  the  WSMR/TC AC -A  T-l  link. 

5.1  Determine  the  “no  load”  latency  using  pings. 

5.2  Determine  the  loaded  latency  using  pings  at  varying  rates. 

5.3  Successful  transmission  of  DIS  PDUs  across  the  link. 

5.4  Determine  the  packet  rate  (packets  per  second)  in  which  DIS  PDUs  are  lost. 

6.  Results: 

6.1.  Data  Recorded. 

6.1.1  No  Load  Latency  Test:  The  first  test  conducted  was  the  no  load  latency  test  using 
pings.  This  test  involved  using  32  pings  for  measuring  the  baseline  round  trip  latency  from  the 
TCAC  to  WSMR.  The  minimum  load  for  these  pings  were  64  bytes.  This  test  yielded  the 
following  results: 


Table  6.1.1.  Latency  Test  -  N 

o  Load 

Date 

Source 

Destination 

No.  Pkts 

Round-trip  Time  (ms) 

Minimum 

Maximum 

Average 

1  Dec  97 

TCACJndy 

WSMR  Indy 

32 

41 

44 

41 

6.1.2.  Loaded  Latency  Test:  The  second  test  consisted  of  transmitting  loaded  pings, 
144  bytes,  at  different  rates  and  packet  sizes  to  determine  round  trip  latencies-  see  table  6.1.2. 
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Table  6.1.2.  Latency  Test  -  Loaded 


Date 

Source 

Destination 

Pkt 

Rate(sec) 

Pkts 

Sent/Rec 

Rounc 

-trip  Time  (ms)  | 

Min 

Max 

Ave 

1  Dec  97 

TCACJndy 

WSMR  Indy 

.01 

320/320 

42 

45 

42 

.005 

640/639 

42 

85 

mm 

.0025 

960/960 

42 

51 

— 

6.1.3.  PDU  Verification  Test:  The  next  test  used  PDUs  generated  at  100  packets  per 
second  from  JANUS  and  recorded  by  the  JADS  logger.  The  PDUs  were  replayed  from  the 
TCAC  and  logged  at  WSMR  -  see  table  6.1.3. 


Table  6.1.3.  PDU  Verification  Test 


Date 

Location 

No.  of  PDUs 
Received 

No.  of  PDUs 
Transmitted 

No.  of  PDUs 
Out-of-order 

No.  of  PDUs 
>  1  sec. 

5  Nov  97 

WSMR 

11859 

11859 

0 

0 

6.1.4.  Stress  Test:  This  test  involved  replaying  the  ESPDU  file  at  its  recorded  excepted 
data  rate  (EDR),  then  increasing  the  EDR  by  100  packets  per  second  incrementally  to  five  times 
the  EDR.  During  the  actual  playback  of  the  PDU  log  fil q,  pings  were  transmitted  and  latency 
statistics  recorded.  Spectrum  also  recorded  the  bandwidth  utilized. 


Table  6.1.4.  Stress  Test 


Rate  (PDU/sec) 

Ping  Time  (ms) 
Min/Max/Ave 

Number  PDUs 
Received 

1  X  E.R. 

100 

59/61/59 

11859 

2  X  E.R. 

200 

41/61/59 

11858 

3  X  E.R. 

41/60/45 

11853 

4  X  E.R. 

400 

41/61/58 

11850 

41/70/56 

11849 

6.2.  Problems:  PDUs  were  lost  during  the  execution  of  the  Stress  Test.  N&E  continues  to 
investigate  packet  drop-outs. 

6.3.  Summary:  During  the  execution  of  the  characterization,  latencies  and  PDU  drop-outs 
were  less  than  anticipated.  Unlike  several  other  T-l  links  characterized  while  using  a  routed 
network  configuration,  WSMR  used  bridging.  This  technique  provided  a  much  more  stable 
environment  thus  allowing  for  lower  transmission  latencies  and  fewer  PDU  losses.  Each 
characterization  test  was  conducted  and  the  results  annotated  in  the  previous  tables.  The 
WSMR/TCAC-A  link  can  support  the  maximum  expected  data  rate  of  400  PDUs  per  second. 

GREGORY  P.  DEWIT 
CPT(P),  FA 
Test  Analyst 
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MEMORANDUM  FOR  RECORD 


26  Feb  98 


FROM  :  JADS  JTF/ETE/N&E 
SUBJECT:  Characterization  Report 

1.  Event  Name:  Fort  Hood/TCAC-A  Annex  T-l  Characterization. 

2.  Date/Time:  2  Feb  98;  0900  -  1 100. 

3.  Sites:  Fort  Hood,  TX;  JADS  TCAC  -  Albuquerque,  NM. 

4.  Description:  The  characterization  involved  base-lining  the  performance  of  the  T-l  link 
between  Fort  Hood  and  the  TCAC -A.  A  variety  of  Networking  and  Engineering  tests  were  used 
to  calculate  the  round  trip  latencies  and  determine  the  maximum  rate  at  which  DIS  PDUs  could  be 
transmitted  before  drop-outs  occurred.  The  tests  used  to  gather  this  data  included  the  No  Load 
Latency  Test,  the  Loaded  Latency  Test,  PDU  Verification  Test,  and  the  Stress  Test. 

5.  Objective:  Assess  (characterize)  the  Fort  Hood/TCAC-A  T-l  link. 

5.1  Determine  the  “no  load”  latency  using  pings. 

5.2  Determine  the  loaded  latency  using  pings  at  varying  rates. 

5.3  Successful  transmission  of  DIS  PDUs  across  the  link. 

5.4  Determine  the  packet  rate  (packets  per  second)  in  which  DIS  PDUs  are  lost. 

6.  Results: 

6.1.  Data  Recorded. 

6.1.1  No  Load  Latency  Test:  The  first  test  conducted  was  the  no  load  latency  test  using 
pings.  This  test  involved  using  32  pings  for  measuring  the  baseline  round  trip  latency  from  the 
TCAC  to  Fort  Hood.  The  minimum  load  for  these  pings  were  64  bytes.  This  test  yielded  the 
following  results: 


Table  6.1.1.  Latency  Test  -  N 

o  Load 

Date 

No.  Pkts 

Round-trip  Time 

(ms) 

Minimum 

Maximum 

Average 

1  Dec  97 

ESSEMBSI 

32 

37 

77 

38 

6.1.2.  Loaded  Latency  Test:  The  second  test  consisted  of  transmitting  loaded  pings,  144 
bytes,  at  several  different  rates  and  packet  sizes  to  determine  the  round  trip  latencies-  see  table 


6.1.2. 
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Table  6.1.2.  Latency  Test  -  Loaded 


Date 

Source 

Destination 

Pkt 

Rate(sec) 

Pkts 

Sent/Rec 

Rounc 

-trip  Time  (ms)  j 

Min 

Max 

Ave 

1  Dec  97 

TCACJndy 

Fort  Hood  Indy 

.01 

320/320 

38 

39 

38 

.005 

640/639 

39 

55 

39 

.0025 

960/960 

38 

76 

39 

6.1.3.  PDU  Verification  Test:  The  next  test  used  PDUs  generated  at  100  packets  per 
second  from  JANUS  and  recorded  by  the  JADS  logger.  The  PDUs  were  replayed  from  the 
TCAC  and  logged  at  Fort  Hood  -  see  table  6.1.3. 


Table  6.1.3.  PDU  Verification  Test 


Date 

Location 

No.  of  PDUs 
Received 

No.  of  PDUs 
Transmitted 

No.  of  PDUs 
Out-of-order 

No.  of  PDUs 
>  1  sec. 

5  Nov  97 

Fort  Hood 

11859 

11859 

0 

6.1.4.  Stress  Test:  This  test  involved  replaying  the  ESPDU  file  at  its  recorded  excepted 
data  rate  (EDR),  then  increasing  the  EDR  by  1 00  packets  per  second  incrementally  to  five  times 
the  EDR.  During  the  actual  playback  of  the  PDU  log  file,  pings  were  transmitted  and  latency 
statistics  recorded.  Spectrum  also  recorded  the  bandwidth  utilized. 


Table  6.1.4.  Stress  Test 


Rate  (PDU/sec) 

Ping  Time  (ms) 
Min/Max/Ave 

Number  PDUs 
Received 

1  X  E.R. 

100 

40/69/59 

11859 

2  X  E.R. 

200 

37/60/42 

11858 

3  X  E.R. 

300 

50/60/59 

11853 

400 

38/61/58 

11859 

5  X  E.R. 

500 

37/61/54 

11859 

6.2.  Problems:  PDU  drop-out  is  a  reoccurring  problem  during  the  Stress  Test.  N&E  is  still 
investigating. 

6.3.  Summary:  The  Fort  Hood/TCAC-A  bridged  link  was  characterized  in  less  than  four 
hours.  The  appropriate  data  was  gathered  and  appears  in  the  tables  mentioned  earlier.  There 
latencies  were  stable  throughout  the  execution  of  each  test.  The  link  can  support  more  than  the 
maximum  expected  data  rate  with  only  a  couple  of  PDUs  dropped.  A  bridged  network  seems  to 
offer  faster  transmission  rates  and  fewer  PDU  drop-outs. 


GREGORY  P.  DEWIT 
CPT(P),  FA 
Test  Analyst 
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MEMORANDUM  FOR  RECORD 


26  Feb  98 


FROM :  JADS  JTF/ETE/N&E 
SUBJECT:  Characterization  Report 

1.  Event  Name:  Fort  Sill/TCAC-A  Annex  T-l  Characterization. 

2.  Date/Time:  2  Feb  98;  1300  -  1600. 

3.  Sites:  Fort  Sill,  OK;  JADS  TCAC  -  Albuquerque,  NM. 

4.  Description:  The  characterization  involved  base-lining  the  performance  of  the  T-l  link 
between  Fort  Sill  and  the  TCAC-A.  A  variety  of  Networking  and  Engineering  tests  were  used  to 
calculate  the  round  trip  latencies  and  determine  the  maximum  rate  at  which  DIS  PDUs  could  be 
transmitted  before  drop-outs  occurred.  The  tests  used  to  gather  this  data  included  the  No  Load 
Latency  Test,  the  Loaded  Latency  Test,  PDU  Verification  Test,  and  the  Stress  Test. 

5.  Objective:  Assess  (characterize)  the  Fort  Sill/TCAC-A  T-l  link. 

5.1  Determine  the  “no  load”  latency  using  pings. 

5.2  Determine  the  loaded  latency  using  pings  at  varying  rates. 

5.3  Successful  transmission  of  DIS  PDUs  across  the  link. 

5.4  Determine  the  packet  rate  (packets  per  second)  in  which  DIS  PDUs  are  lost. 

6.  Results: 

6.1.  Data  Recorded. 

6.1.1  No  Load  Latency  Test:  The  first  test  conducted  was  the  no  load  latency  test  using 
pings.  This  test  involved  using  32  pings  for  measuring  the  baseline  round  trip  latency  from  the 
TCAC  to  Fort  Sill.  The  minimum  load  for  these  pings  were  64  bytes.  This  test  yielded  the 
following  results: 


Table  6.1.1.  Latency  Test  -  No  Load 


Source 

No.  Pkts 

Round-trip  Time  (ms)  [ 

Minimum 

Maximum 

TCACJndy 

32 

30 

40 

hhi 

6.1.2.  Loaded  Latency  Test:  The  second  test  consisted  of  transmitting  loaded  pings, 
144  bytes,  at  several  different  rates  and  packet  sizes  to  determine  the  round  trip  latencies-  see 
table  6.1.2. 
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Table  6.1.2.  Latency  Test  -  Loaded 


Date 

Source 

Destination 

Pkt 

Rate(sec) 

Pkts 

Sent/Rec 

Rounc 

-trip  Time  (ms)  j 

Min 

Max 

Ave 

1  Dec  97 

TCACJndy 

Indy 

.01 

320/320 

31 

33 

31 

.005 

640/639 

31 

33 

31 

.0025 

960/960 

31 

33 

31 

6.1.3.  PDU  Verification  Test:  The  next  test  used  PDUs  generated  at  100  packets  per 
second  from  JANUS  and  recorded  by  the  JADS  logger.  The  PDUs  were  replayed  from  the 
TCAC  and  logged  at  Fort  Sill  -  see  table  6.1.3. 


Table  6.1.3.  PDU  Verification  Test 


Date 

Location 

No.  of  PDUs 
Received 

No.  of  PDUs 
Transmitted 

No.  of  PDUs 
Out-of-order 

No.  of  PDUs 
>  1  sec. 

5  Nov  97 

Fort  Sill 

11859 

11859 

0 

0 

6.1.4.  Stress  Test:  This  test  involved  replaying  the  ESPDU  file  at  its  recorded  excepted 
data  rate  (EDR),  then  increasing  the  EDR  by  100  packets  per  second  incrementally  to  five  times 
the  EDR.  During  the  actual  playback  of  the  PDU  log  file,  pings  were  transmitted  and  latency 
statistics  recorded.  Spectrum  also  recorded  the  bandwidth  utilized. 


Table  6.1.4.  Stress  Test 


Rate  (PDU/sec) 

Ping  Time  (ms) 
Min/Max/Ave 

Number  PDUs 
Received 

1  X  E.R. 

100 

30/61/58 

11859 

2  X  E.R. 

200 

30/60/52 

11857 

3  X  E.R. 

30/61/43 

11853 

■ 

30/60/36 

11859 

5  X  E.R. 

500 

30/84/300 

10656 

6.2.  Problems:  PDUs  were  lost  during  the  execution  of  the  Stress  Test.  N&E  will  continue 
to  investigate  packet  drop-outs. 

6.3.  Summary:  The  Fort  Sill/TCAC-A  link  involved  using  a  router  to  transmit  PDU  packets. 
The  latencies  were  reasonably  close  to  the  values  during  the  Grumman  Characterization  given  the 
shorter  distance  between  Albuquerque  and  Fort  Sill.  There  were  a  couple  of  packets  lost  while 
incrementing  up  to  four  times  the  EDR.  The  number  of  lost  PDUs  increased  to  about  10  percent 
of  the  packets  transmitted  at  five  times  the  EDR  (500  packets  per  second). 


GREGORY  P.  DEWIT 
CPT(P),  FA 
Test  Analyst 
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SUMMARY:  This  is  a  report  of  the  performance,  validation,  and  verification  testing 
conducted  on  the  LWX  and  external  Cisco  router  platforms  at  the  JADS-JTF  facility, 
Albuquerque,  NM.  This  document  covers  the  steps  taken  during  testing,  the  test  results, 
conclusions  drawn,  and  a  discussion  of  the  alternative  solutions  for  the  broadcasting  of  User 
Datagram  Protocol  (UDP)  traffic  through  the  JADS  JTF  End-to-End  (ETE)  network. 


Table  of  Contents 


1.  Background . 4 

2.  Scope  of  Testing . 4 

3.  Description  of  PX-3  and  LWX  Rel.  3.01 . 4 

4.  Test  Configurations . 5 

4.1  Point  to  Point  PX-3 . 5 

4.2  Point  to  Point  PX-3s  to  APX . 6 

4.3  Cisco  4000  to  PX-3 . 6 

4.4  Cisco  7000  to  PX-3 . 7 

4.5  Cisco  7000  to  4000 . 7 

4.6  Multipoint  UDP  Forwarding  &  Hybrid  Bridge/Router . 7 

5.  Test  Expectations . 9 

6.  Performance  Test  Results  (Point  to  Point  -  UDP  forwarding) . 9 

7.  Performance  Test  Results  (Multipoint  -  UDP  Forwarding) . 11 

8.  Hybrid  Bridge/Router  Configuration . 12 

9.  Conclusions . 13 

10.  Recommendations .  14 

APPENDIX  A.  ACRONYMS/DEFINITIONS . 16 

Figures 

Figure  4-1  -  JADS  ETE  Network  Configuration . 5 

Figure  4.1-1  -  PX-3  to  PX-3  Configuration . 5 

Figure  4.2-1  -  PX-3  to  APX  Configuration . 6 

Figure  4.3-1  -  Cisco  4000  to  PX-3  Configuration . 6 

Figure  4.4-1  -  Cisco  7000  to  PX-3  Configuration . 7 

Figure  4.5-1  -  Cisco  7000  to  Cisco  4000  Configuration . 7 

Figure  4.6-1  -  Multipoint  UDP  Forwarding  &  Hybrid  Bridge/Router  Configuration . 8 


1.  Background 


The  Joint  Advanced  Distributed  Simulation  Joint  Test  Force  (JADS  JTF)  has  built  a  network  of 
seven  nodes  to  support  an  environment  linking  multiple  simulation  sites  whose  primary 
application  is  the  broadcasting  of  User  Datagram  Protocol  packets  from  multiple  sites  to  multiple 
sites.  The  network  supports  the  evaluation  of  Advanced  Distributed  Simulation  (ADS)  for  the 
Department  of  Defense  (DOD)  Test  and  Evaluation  (T&E)  Community.  The  system  is  called  the 
End-to-End  Test  Network  and  the  system  under  test  is  the  Joint  Surveillance  and  Targeting 
Attack  Radar  System  (JSTARS)  and  its  associated  command  and  control  functions.  The  services 
carried  by  this  network  include  the  router  traffic  and  voice  traffic. 

The  goal  of  the  testing  conducted  December  15-16,  1997  was  to  provide  answers  to  whether  the 
PX  platforms  (PX-3,  PX-2,  and  APX)  provided  by  NET  could  support  the  demands  of  the  UDP 
traffic  load  through  the  JADS  network. 

2.  Scope  of  Testing 

1 .  Verification/validation  of  the  problems  witnessed  by  TSGT  Ashton 

2.  Comparison  of  the  performance  gains/losses  of  external  router  configurations 

3.  Effects  of  altering  internal  software  configuration  (buffers,  hold  queues,  etc.) 

4.  Effects  of  using  a  hybrid  routed/bridged  network 

5.  Effects  of  a  completely  bridged  network 

3.  Description  of  PX-3  and  LWX  Rel.  3.01 

The  LAN/WAN  Exchange  (LWX)  Release  3.01  is  a  general  purpose  router/bridge  integrated  into 
the  Integrated  Digital  Network  Exchange  (IDNX)  platform.  It  provides  internetwork  connectivity 
between  Local  Area  Networks  (LANs)  over  Wide  Area  Networks  (WANs)  through  IDNX/  90, 
/70,  /20,  and  /Micro  20  IDNX  nodes.  The  LWX  3.01,  consists  of  a  PX3  front  card  (revision  B  or 
later)  and  an  Ethernet  or  Token  Ring  interface  rear  card,  and  provides: 

—  concurrent  multi-protocol  routing  and  fallback  Media  Access  Control  (MAC)  layer  bridging 

—  up  to  8  logical  serial  connections  to  other  LWXs  or  external  routers 

—  an  Ethernet  interface  rear  card 

The  first  phase  of  LWX  3.01  only  supports  four  and  eight  port  PX3  cards.  Earlier  LWX  versions 
consisted  of  a  PX-  2,  PX-  Plus,  or  Access  PX  front  card  and  an  Ethernet  or  Token  Ring  interface 
rear  card,  and  provide  only  eight  logical  serial  connections  to  other  LWXs  or  external  routers. 
LWX  Release  3.01  supports  only  Ethernet  and  Token  Ring  (available  in  a  later  release)  interface 
cards.  LWX  Release  3.01  must  use  a  revision  B  (or  later)  PX3  card. 

The  JADS  network  uses  four  port  PX3,  eight  port  PX-2,  and  four  port  APX  cards.  The  PX-3s 
run  LWX  Release  3.01  and  the  PX-2s  and  APXs  run  LWX  Release  2.02.06.  The  LWX  Release 
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3.01  is  equivalent  to  Cisco’s  IOS  Release  11.0(9).  The  LWX  Release  2.02.06  is  equivalent  to 

4.  Test  Configurations 


Classified  Unclassified 


Figure  4-1  -  JADS  ETE  Network  Configuration 


Figure  4-1  above  illustrates  a  general  configuration  of  the  JADS  ETE  network.  The  connectivity 
between  the  nodes  remained  consistent  throughout  the  testing.  However,  card  and  port 
configurations  were  varied  for  the  different  tests. 

4.1  Point  to  Point  PX-3 

IDNX-20  IDNX-20 

w/  PX-3  w/  PX-3 


Figure  4.1-1  -  PX-3  to  PX-3  Configuration 


Figure  4.1-1  above  illustrates  the  setup  used  for  the  verification/validation  using  two  PX-3s  point- 
to-point. 

4.2  Point  to  Point  PX-3s  to  APX 


IDNX-20  IDNX-20 

w/  PX-3  w/  APX 


Broadcaster 


liSiil 


.3  44  Mh/s 


m 

Logger 


Figure  4.2-1  -  PX-3  to  APX  Configuration 


Figure  4.2-1  above  illustrates  the  setup  used  for  the  verification/validation  using  one  PX-3  and 
one  APX  point-to-point. 

4.3  Cisco  4000  to  PX-3 


IDNX-20  IDNX-20 

w/  HSD-2  w/  PX-3 


Broadcaster  Logger 


Figure  4.3-1  -  Cisco  4000  to  PX-3  Configuration 

Figure  4.3-1  above  illustrates  the  setup  used  for  the  verification/validation  using  an  externally 
connected  Cisco  4000  running  Cisco  IOS  Release  9.1  to  a  PX-3  point-to-point.  The  Cisco  4000 
was  serially  connected  through  an  HSD-2  port  on  the  IDNX.  The  HSD-2  port  was  mapped  to 
the  distant  end  PX-3. 
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4.4  Cisco  7000  to  PX-3 


Broadcaster 


Cisco  7000 


IDNX-20 
w/  HSD-2 


1.344  Mb/s 


IDNX-20 
w/  PX-3 


Logger 


Figure  4.4-1  -  Cisco  7000  to  PX-3  Configuration 

Figure  4.4-1  above  illustrates  the  setup  used  for  the  verification/validation  using  an  externally 
connected  Cisco  7000  running  Cisco  IOS  Release  1 1.x  to  a  PX-3  point-to-point.  The  Cisco  7000 
was  serially  connected  through  an  HSD-2  port  on  the  IDNX.  The  HSD-2  port  was  mapped  to 
the  distant  end  PX-3. 


4.5  Cisco  7000  to  4000 

Cisco  7000 


IDNX-20 
w/  HSD-2 
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Logger 


Figure  4.5-1  -  Cisco  7000  to  Cisco  4000  Configuration 

Figure  4.5-1  above  illustrates  the  setup  used  for  the  verification/validation  using  an  externally 
connected  Cisco  7000  running  Cisco  IOS  Release  11.x  to  another  externally  connected  Cisco 
4000  point-to-point.  Both  Cisco  routers  were  serially  connected  through  HSD-2  ports  on  the 
IDNX.  The  HSD-2  ports  were  mapped  end  to  end. 

4.6  Multipoint  UDP  Forwarding  &  Hybrid  Bridge/Router 


IDNX-20 
w/  PX-3 


Broadcaster 


Broadcaster 


Figure  4.6-1  -  Multipoint  UDP  Forwarding  &  Hybrid  Bridge/Router  Configuration 


Figure  4.6-1  above  illustrates  the  setup  used  for  the  verification/validation  using  multiple  sources 
to  broadcast  DIS  Entity  State  PDUs  to  a  central  logger.  This  configuration  represents  a  portion 
of  the  unclassified  side  of  the  JADS  ETE  network. 
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5.  Test  Expectations 


Throughout  all  the  tests  the  Broadcaster,  an  SGI  Indy  workstation,  used  a  file  that  consisted  of 
11,859  Distributed  Interactive  Simulation  (DIS)  Entity  State  protocol  data  units  (PDUs).  The 
SGI  workstation  broadcasted  the  PDUs  on  its  local  area  network  (LAN)  as  User  Datagram 
Protocol  (UDP)  packets.  The  LWXs  were  configured  to  forward  these  broadcasts  through  the 
network.  Each  Entity  State  PDU  was  144  bytes  in  length.  The  Internet  Protocol  (IP)  datagram 
produced  was  152  bytes  in  length.  All  this  was  verified  using  a  Network  General  Sniffer  Protocol 
Analyzer  on  the  Broadcaster  subnetwork. 

At  the  distant  end,  the  Logger,  also  an  SGI  Indy  workstation,  listened  for  UDP  broadcasts  and 
counted  the  number  received.  The  difference  is  the  number  of  packets  dropped  within  the 
network. 

We  expected  to  see  a  threshold  that  each  platform  would  reach  in  terms  of  performance.  We 
knew  that  the  unreliable  nature  of  the  traffic  (UDP  broadcasts)  would  force  the  routers  to  inspect 
each  packet  since  it  was  a  broadcast,  and  forward  it  onto  each  interface.  This  activity  is  known  as 
process  switching.  The  fact  that  every  packet  sent  is  process  switched,  we  knew  there  had  to  be  a 
limit  to  the  number  of  packets  successfully  received  when  a  workstation  floods  the  network.  We 
also  expected  to  see  very  little  improvement  when  changing  the  buffers  and  hold-queues.  The  fact 
remained  that  if  packets  are  process-switched,  there  is  a  finite  limit  to  speed  in  which  it  forwards 
packets  successfully.  We  will  discuss  more  about  these  issues  in  our  conclusions. 

The  following  two  sections  cover  the  result  sets  for  the  two  separate  environments  (point-to-point 
and  multipoint).  The  point-to-point  environment  results  were  used  to  gauge  the  expected  results 
when  we  combined  multiple  stations  broadcasting  in  the  network.  The  multipoint  configuration 
better  represents  the  true  network  that  will  be  fielded  to  support  the  JADS  ETE  network. 

6.  Performance  Test  Results  (Point  to  Point  -  UDP  forwarding) 

Broadcaster  generating  400  packets  per  second  - 


Configuration 

Packets  Dropped  Packets  Dropped 

(Broadcast  Side j  (Logger  Side) 

PX-3  to  PX-3 

0 

0 

PX-3  to  APX 

0 

o 

Cisco  4000  to  PX-3 

0 

0 

Cisco  7000  to  PX-3 

0 

0 

Cisco  4000  to  Cisco  7000 

0 

0 

Cisco  7000  to  Cisco  4000 

0 

0 
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Broadcaster  generating  500  packets  per  second 


Configuration 

Packets  Dropped  Packets  Dropped 

(Broadcast  Side)  (Logger  Side) 

PX-3  to  PX-3 

2366 

0 

PX-3  to  APX 

2366 

0 

Cisco  4000  to  PX-3 

0 

2366 

Cisco  7000  to  PX-3 

0 

2366 

Cisco  4000  to  Cisco  7000 

0 

0 

Cisco  7000  to  Cisco  4000 

0 

0 

Broadcaster  generating  600  packets  per  second 


Configuration 

Packets  Dropped 
(Broadcast  Side) 

Packets  Dropped 

(Logger  Side) 

PX-3  to  PX-3 

3927 

0 

PX-3  to  APX 

3927 

0 

Cisco  4000  to  PX-3 

0 

3927 

Cisco  7000  to  PX-3 

0 

3927 

Cisco  4000  to  Cisco  7000 

0 

0 

Cisco  7000  to  Cisco  4000 

0 

0 

Broadcaster  generating  700  packets  per  second 


Configuration 

Packets  Dropped  Packets  Dropped 

(Broadcast  Side)  (Logger  Side) 

PX-3  to  PX-3 

4722 

0 

PX-3  to  APX 

4722 

0 

Cisco  4000  to  PX-3 

0 

4722 

Cisco  7000  to  PX-3 

0 

4722 

Cisco  4000  to  Cisco  7000 

0 

0 

Cisco  7000  to  Cisco  4000 

0 

0 
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Broadcaster  generating  800  packets  per  second 


Configuration 

Packets  Dropped 
(Broadcast  Side) 

Packets  Dropped 

(Logger  Side) 

PX-3  to  PX-3 

5646 

0 

PX-3  to  APX 

5646 

0 

Cisco  4000  to  PX-3 

623 

5023 

Cisco  7000  to  PX-3 

0 

5646 

Cisco  4000  to  Cisco  7000 

623 

0 

Cisco  7000  to  Cisco  4000 

0 

0 

Broadcaster  generating  900  packets  per  second 


Configuration 

Packets  Dropped 
(Broadcast  Side) 

Packets  Dropped 

(Logger  Side) 

PX-3  to  PX-3 

6239 

0 

PX-3  to  APX 

6239 

0 

Cisco  4000  to  PX-3 

1717 

4522 

Cisco  7000  to  PX-3 

276 

5963 

Cisco  4000  to  Cisco  7000 

1717 

0 

Cisco  7000  to  Cisco  4000 

276 

0 

7.  Performance  Test  Results  (Multipoint  -  UDP  Forwarding) 

Due  to  the  limited  resources  and  the  overall  intention  of  the  testing,  we  restricted  the  multipoint 
tests  to  the  PX  platforms.  The  question  to  be  answered  was  whether  the  PX-3  would  respond  the 
same  way  as  the  point  to  point  test  but  with  various  interfaces  transmitting  broadcast  traffic.  With 
two  broadcasting  spoke  sites  and  a  single  logger  hub  site  we  got  the  following  results: 


Ell 


Broadcaster  generating  400  packets  per  second  (aggregate)  -  one  site  transmitting  at  200  pps  and 
a  second  site  transmitting  at  200  pps. 


Configuration 

Packets  Dropped  Packets  Dropped 

(Broadcast  Side)  (Logger  Side) 

PX-3  Hub  /  PX-3  Spokes 

0 

0 

PX-3  Hub/  APX  Spokes 

0 

0 

Cisco  4000  to  PX-3 

Not  Tested 

Not  Tested 

Cisco  7000  to  PX-3 

Not  Tested 

Not  Tested 

Cisco  7000  to  Cisco  4000 

Not  Tested 

Not  Tested 

Broadcaster  generating  500  packets  per  second  (aggregate)  -  one  site  transmitting  at  300  pps  and 
a  second  site  transmitting  at  200  pps. 


Configuration 

Packets  Dropped  Packets  Dropped 

(Broadcast  Side)  (Logger  Side) 

PX-3  Hub  /  PX-3  Spokes 

0 

2366 

PX-3  Hub/  APX  Spokes 

0 

2366 

Cisco  4000  to  PX-3 

Not  Tested 

Not  Tested 

Cisco  7000  to  PX-3 

Not  Tested 

Not  Tested 

Cisco  7000  to  Cisco  4000 

Not  Tested 

Not  Tested 

As  expected,  the  aggregate  broadcast  traffic  at  the  hub  site  creates  the  same  results.  The  total 
process-switching  capacity  for  the  PX  platform  is  limited  and  independent  of  the  interfaces  used. 

8.  Hybrid  Bridge/Router  Configuration 

One  configuration  that  we  were  able  to  test  was  a  hybrid  bridge/router  configuration.  Our  testing 
was  restricted  to  the  unclassified  portion  of  the  network.  In  this  configuration  the  two  spoke 
locations  bridged  the  serial  and  Ethernet  interfaces.  In  the  hub  location,  we  utilized  bridging  on 
the  unclassified  interfaces  and  routing  for  the  serial  interface  going  over  to  the  classified  portion 
of  the  network.  With  this  configuration  we  were  able  to  successfully  pass  an  aggregate  of  600 
pps  to  the  logger  off  the  ethemet  interface  of  the  hub  node.  The  same  file  was  broadcast  from 
two  different  bridge  segments,  so  the  total  number  of  PDUs  broadcast  was  23718  packets. 
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Broadcaster  generating  600  packets  per  second  (aggregate)  -  one  site  transmitting  at  300  pps  and 
a  second  site  transmitting  at  300  pps. 


Configuration 

Packets  Dropped  Packets  Dropped 

(Broadcast  Side)  (Logger  Side) 

PX-3  Hub  /  PX-3  Spokes 

0 

0 

PX-3  Hub/  APX  Spokes 

0 

0 

Cisco  4000  to  PX-3 

Not  Tested 

Not  Tested- 

Cisco  7000  to  PX-3 

Not  Tested 

Not  Tested 

Cisco  4000  to  Cisco  7000 

Not  Tested 

Not  Tested 

Cisco  7000  to  Cisco  4000 

Not  Tested 

Not  Tested 

Broadcaster  generating  700  packets  per  second  (aggregate)  -  one  site  transmitting  at  300  pps  and 
a  second  site  transmitting  at  400  pps. 


Configuration 

Packets  Dropped  Packets  Dropped 

(Broadcast  Side)  (Logger  Side) 

PX-3  Hub  /  PX-3  Spokes 

0 

2594  of  237 18  sent 

PX-3  Hub/  APX  Spokes 

0 

2594  of  23718  sent 

Cisco  4000  to  PX-3 

Not  Tested 

Not  Tested 

Cisco  7000  to  PX-3 

Not  Tested 

Not  Tested  j 

Cisco  4000  to  Cisco  7000 

Not  Tested 

Not  Tested 

Cisco  7000  to  Cisco  4000 

Not  Tested 

Not  Tested 

Further  testing  must  be  done  to  verify  the  effects  on  the  traffic  sent  over  the  serial  connection  to 

the  classified  logger. 

9.  Conclusions 

•  The  limitations  uncovered  during  the  testing  involved  the  level  of  process  switching  capacity 
on  the  PX-3,  the  Cisco  4000,  and  the  Cisco  7000. 

•  The  use  of  UDP  -  an  inherently  unreliable  transport  mechanism  -  is  not  always  a  suitable 
choice  of  transport  for  data  that  needs  a  high  level  of  reliability.  The  transmission  control 
protocol  (TCP)  is  better  suited  for  reliable  transport  since  it  uses  sequencing  and 
acknowledgments,  but  at  a  cost  of  increased  latency  -  which  was  not  tested.  Also,  the  use  of 
multicasting  or  unicasting  would  take  advantage  of  the  fast  switching  capability  of  all  the 
routers  tested. 
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•  The  use  of  broadcast  based  networking  has  an  adverse  effect  on  the  network.  Routing  this 
traffic  adds  one  additional  layer  of  processing  (especially  when  the  broadcasts  are  process 
switched)  and  creates  multiple  copies  of  each  datagram  in  order  to  forward  it  to  multiple 
network  subnets.  Thus,  congesting  the  network  with  broadcast  traffic.  Since  the  key 
application  relies  on  broadcasts,  we  should  consider  flattening  the  network  and  use  bridging. 

•  There  were  definitive  “break  points”  where  the  processors  could  not  handle  anymore  packets. 
This  is  true  on  every  platform  tested. 

•  During  the  testing,  we  did  notice  that  there  were  drops  on  the  hold-queue,  missed  buffer 
requests,  and  fallbacks  on  the  interface  buffers.  To  remedy  this  we  added  to  the  hold-queue 
length  and  increased  the  number  of  permanent  big  buffers.  The  actual  number  of  successful 
packets  sent  never  rose  above  the  initial  ceiling.  In  other  words,  the  addition  of  buffers  and 
increasing  the  hold-queue  did  not  affect,  in  any  way,  the  speed  at  which  the  processor 
process-switched  the  packets.  This  was  expected. 

•  It  is  our  best  judgment,  the  limitation  is  with  the  router’s  processor  handling  UDP  broadcasts. 
With  UDP  broadcast  traffic,  all  packets  are  process  switched.  With  unicast  and  multicast 
traffic,  the  router  is  capable  of  fast  switching  most  of  the  packets.  Routers  gain  efficiencies 
when  they  are  capable  of  caching  routes  for  packets. 

•  We  had  discussions  with  Technical  Assistance  Center  Staff  Engineers  from  both  NET  and 
Cisco  Systems  over  the  last  two  weeks.  Engineers  from  both  companies  stated  that  none  of 
their  router  platforms  have  ever  been  tested  to  find  what  are  the  limits  of  reliable  transport  of 
UDP  packets. 

•  The  overall  configuration  relies  on  the  broadcaster.  If  the  network  has  to  reliably  transport 
UDP  broadcasts  above  400  pps  (aggregate)  to  the  hub  node  to  be  logged  then  a  PX-3/APX 
routed  solution  will  not  suffice. 

10.  Recommendations 

The  initial  requirements  of  the  JADS  ETE  network  included  a  broadcaster  transmitting  100 
packets  per  second.  All  of  the  configurations  tested  will  satisfy  this  requirement.  Only  when  the 
aggregate  packet  load  on  the  PX-3  (using  UDP  forwarding)  exceeded  400  packets  per  second 
was  there  a  concern  for  lost/dropped  packets.  With  bridging,  this  ceiling  was  up  to  800  packets 
per  second.  If  the  JADS  ETE  network  is  to  continue  using  broadcasted  UDP  packets  with  a 
small  number  of  sites,  bridging  would  be  a  better  solution.  Options  and  limitations  are  as  follows: 

Routing 

PX-3/APXs  -  400  pps  aggregate  limitation 

External  Cisco  4000  Routers  at  each  site  (serially  connected  via  HSD-2s)  -  up  to  700  pps 
aggregate  limitation;  higher  cost 
Bridging 
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PX-3/APXs  -  800  pps  aggregate  limitation 

Hybrid  Routing/Bridging 

PX-3/APXs  -  600  pps  aggregate  limitation 

Several  issues  have  to  be  addressed  (weighing  the  pros  and  cons): 

•  Broadcasting  at  lower  rates. 

•  Pro:  Limiting  the  speed  of  the  broadcasts  to  under  a  400  pps  aggregate  would  ensure  the 
ability  to  forward  all  of  the  UDP  traffic  through  the  network. 

•  Con:  The  impact  of  slower  rates  on  the  simulation  is  unknown. 

•  The  use  of  multicasting  in  the  future. 

•  Pro:  The  use  of  multicasting  would  take  advantage  of  the  router’s  ability  to  fast  switch 
the  traffic.  The  router  only  has  to  process  switch  the  first  packet  to  find  the  route  and  will 
stream  the  following  data  along  the  same  route. 

•  Con:  Finding  a  suitable  data  logger  may  be  difficult  or  a  logger  would  have  to  be  written, 
and  the  impact  on  the  simulation  is  unknown. 

•  Bridging  the  entire  network. 

•  Pro:  The  use  of  bridging  would  increase  the  reliable  throughput  of  UDP  traffic  to 
approximately  800  pps.  However,  this  option  should  be  further  tested  on  the  entire  JADS 
ETE  network. 

•  Con:  Speed  of  transmission  is  higher,  but  still  limited  to  800  pps.  Also,  it  may  be 
difficult  to  persuade  all  sites  to  be  part  of  the  same  subnet. 

•  The  cost  of  using  external  routers. 

•  Pro:  Different  router  platforms  have  different  processing  power.  There  are  other 
platforms  that  have  better  performance  in  forwarding  UDP  packets. 

•  Con:  There  are  tangible  limitations  where  the  processors  could  not  handle  process 
switched  traffic  -  regardless  of  platform.  Candidate  platforms  should  be  tested  for  this 
application  before  procurement.  Also,  cost  is  a  factor.  Costing  information  is  unavailable 
at  this  time,  but  it  is  safe  to  state  that  the  more  powerful  the  processor,  the  higher  the  cost. 
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APPENDIX  A.  ACRONYMS/DEFINITIONS 


ADS 

Advanced  Distributed  Simulation 

DIS 

Distributed  Interactive  Simulation 

DOD 

Department  of  Defense 

IP 

Internet  Protocol:  The  network  layer  for  the  TCP/IP  Protocol  Suite. 

JADS  JTF 

JOINT  ADVANCED  DISTRIBUTED  SIMULATION  JOINT  TEST  FORCE 

JSTARS 

Joint  Surveillance  Targeting  and  Attack  Radar  System 

LAN 

Local  Area  Network 

MAC 

Media  Access  Control:  The  lower  portion  of  the  datalink  layer. 

MAC-Layer  Bridge 

A  device  that  connects  two  or  more  similar  networks  in  a  way  that  is 
transparent  to  the  users  of  the  network  layer. 

OSI 

Open  Systems  Interconnection:  The  seven  layer  suite  of  protocols  to  be 
the  international  standard  computer  network  architecture. 

PDU 

Protocol  Data  Unit:  An  OSI  term  for  “packet.”  A  PDU  is  a  data  object 
exchanged  by  protocol  machines  (entities)  within  a  given  layer. 

PPS 

Packets  Per  Second 

T&E 

Test  and  Evaluation 

TCP 

Transmission  Control  Protocol:  An  Internet  standard  transport  protocol 
in  the  Internet  protocols  suite  for  reliable,  connection-oriented,  full-duplex 
streams. 

UDP 

USER  DATAGRAM  PROTOCOL:  An  Internet  standard  transport  protocol 
that  exchanges  datagrams  without  acknowledgments  or  guaranteed  delivery. 

WAN 

Wide  Area  Network 

All  Definitions  are  from  Guide  to  Networking  and  Internetworking  Terms.  Simoneau,  Paul,  ©  1993,  American  Group,  Cary  NC 
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Appendix  F 
POWERSIM™ 


The  software  chosen  to  develop  the  simulation  of  the  detailed  design  of  the  End-to-End  (ETE) 
Test  synthetic  environment  (SE)  is  called  POWERSIM™,  a  dynamic  simulation  tool  developed  by 
ModellData  AS,  Bergen,  Norway.  Many  other  software  tools  exist  which  may  be  used  to  develop 
simulations  of  detailed  designs.  What  is  important  is  that  the  software  allow  the  modeling  of  all 
kinds  of  dynamic  processes  and  be  easily  adaptable  to  change,  so  that  variations  of  the  concept 
may  be  easily  explored.  POWERSIM™  allows  you  to  start  out  with  a  holistic  causal-loop  view  of 
a  problem  and  end  up  with  a  concise  flow  diagram  model  of  the  same  problem,  ready  to  be 
simulated  over  time.  Figure  F-l  is  a  portion  of  the  graphical  diagram  of  the  simulation  developed 
for  the  entity  state  protocol  data  units  flow  within  the  JADS  ETE  SE. 


As  can  be  seen  from  the  graphical  diagram,  variables,  constants,  flowrates,  and  flow  variables  are 
used  to  develop  a  model  of  the  JADS  ETE  SE.  This  simulation  can  be  run  over  a  definable  time 
period,  with  the  final  values  shown  in  the  small  boxes  over  each  element  of  the  model.  As  each 
time  step  is  executed,  the  current  values  are  shown  above  the  model  element.  The  larger  boxes 
beside  certain  model  elements  allow  for  easy  changing  of  constants  between  simulation  runs.  All 
variables,  to  include  flow  rates  and  flow  variables,  are  user  definable. 


POWERSIM™  also  allows  the  user  to  use  Boolean  variables,  which  can  alarm  when  the  value  of 
the  variable  is  false,  and  allows  the  plotting  of  the  values  of  a  variable  for  each  time  step.  This  is 
illustrated  in  Figure  F-2. 


Figure  F-2.  Graphical  Plots  of  Variable  Values 


In  addition,  the  values  over  time  of  variables  within  the  simulation  can  be  plotted  as  separate  plots 
available  for  export  to  other  software  packages.  This  is  shown  in  Figure  F-3. 


Figure  F-3.  Variable  Plots 


Finally,  POWERSIM™  allows  the  values  of  variables  to  be  displayed  in  tables,  ready  for  export 
into  other  software  packages  for  further  analysis.  This  is  shown  in  Figure  F-4. 


TIME  [TOTAL  MOVERS'  ES  PDU  RATE  |  ENTITIES  STOPPING 


359  _ 681,00  _ 46.00 _ 14.00 

360  ' _ 670.00"  ~  "45.00  _ 14.00 

36  r _  659.00  _ 44.00  _ 14  TOO 

362_  " _ _J48.00] _ 4400  f _ j 3.00 

363  _  638.00  f  "  43T00  ]  ___  13.00 

364'  ~  628.00] .  43.00|  '  13.00 

365  "  _  _  618.00] _ 42.00  _ _  1000 

366  .  600  00T _  -42700  13.00 


367  _ _ 59000  _  _ _ 41.00 _ 12.00 

"368"  ’ _ 589.00]  __] _ 41.00  ~ _ 12.00 

369  _ _ ‘58000  _ _ 40.00  _ 12.00 

*370  5717001  4000] _ 12,00" 

371  562.00";  40.00 _ 12.00 


Figure  F-4.  Variable  Tables 
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Appendix  G 

Acronyms  and  Definitions 


AC 

ACE 

ADS 

AOI 

ARIES 

ASAS 

ATACMS 

BIU 

CDRL 

CGS 

dB 

DIS 

DoD 

ESPDU 

ETE 

FID 

GD 

GDT 

GFE 

GNIU 

GRCA 

GSM 

GSMR 

Hz 

ID 

IDNX 

IEEE 

IP 

IPT 

JADS 


Janus 

Joint  STARS 

JTF 

LAN 

LGSM 

LWX 

M&S 

MAC 

MTI 


attack  control 

analysis  and  control  element;  AMRAAM  captive  equipment  (a  pod  that 
simulates  an  AMRAAM) 
advanced  distributed  simulation 
area  of  interest 

Advanced  Radar  Imaging  Emulation  System 

All  Source  Analysis  System 

Army  Tactical  Missile  System 

bus  interface  unit 

contract  data  requirements  list 

common  ground  station;  Continental  United  States  ground  station 
decibel 

distributed  interactive  simulation 
Department  of  Defense 
entity  state  protocol  data  unit 
End-To-End  Test 
feature  identification 
graphics  display 
ground  data  terminal 
government  furnished  equipment 
ground  network  interface  unit 
ground  reference  coverage  area 
ground  station  module 

Ground  Station  Module  Replicator,  Fort  Huachuca,  Arizona 
hertz 

identification 

Integrated  Digital  Network  Exchange 
Institute  of  Electrical  and  Electronics  Engineers 
internet  protocol;  initial  point 
integrated  product  team 

joint  advanced  distributed  simulation  or  Joint  Advanced  Distributed 

Simulation,  Albuquerque,  New  Mexico 

interactive,  computer-based  simulation  of  combat  operations 

Joint  Surveillance  Target  Attack  Radar  System 

joint  test  force  or  Joint  Test  Force,  Albuquerque,  New  Mexico 

local  area  network 

light  ground  support  module 

local  area  network/wide  area  network  (LAN/W AN)  Exchange 
modeling  and  simulation 
media  access  control 
moving  target  indicator 
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N&E 

network  and  engineering 

NET 

network  equipment  technologies 

NIU 

network  interface  unit 

OSD 

Office  of  the  Secretary  of  Defense 

OSI 

open  systems  inteconnection 

PDU 

protocol  data  unit 

PFA 

probability  of  false  alarm 

Pg 

probability  of  successful  guidance 

POWERSIM™ 

a  dynamic  simulation  developed  by  ModellDATA  AS,  Bergen,  Norway 

PPS 

packets  per  second 

PSP 

programmable  signal  processor 

PTD 

program  test  design 

pip 

program  test  plan 

RDP 

radar  data  processor 

RPSI 

radar  processor  simulator  and  integrator 

RSR 

radar  service  request 

SAR 

synthetic  aperture  radar 

SCDL 

surveillance  control  data  link 

SE 

synthetic  environment 

SGI 

Silicon  Graphics,  Inc. 

SOW 

statement  of  work 

STP 

software  test  plan 

STR 

software  trouble  report 

STRICOM 

U.S.  Army  Simulation,  Training,  and  Instrumentation  Command 

T&E 

test  and  evaluation 

T-l 

digital  carrier  used  to  transmit  a  formatted  digital  signal  at  1 .544  megabits 
per  second 

TAFSM 

Tactical  Army  Fire  Support  Model 

TCAC 

Test  Control  and  Analysis  Center,  Albuquerque,  New  Mexico 

TCP 

transmission  control  protocol 

TOA 

time  of  arrival 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 

UAV 

unmanned  aerial  vehicle 

UDA 

user  defined  activity  area 

UDP 

user  datagram  protocol 

V&V 

verification  and  validation 

VSTARS 

Virtual  Surveillance  Target  Attack  Radar  System 

VV&A 

verification,  validation,  and  accreditation 

WAN 

wide  area  network 

WSMR 

White  Sands  Missile  Range,  New  Mexico 

PHASE  2 

VERIFICATION  AND  VALIDATION  REPORT 
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1.0  Scope 


This  report  provides  the  results  of  the  verification  and  validation  (V&V)  tasks  performed  during 
Phase  2  of  the  End-To-End  (ETE)  Test 

1.1  Purpose 

This  report  details  the  results  of  executing  the  V&V  requirements  listed  within  the  ETE  Test 
Activity  Plan  Appendix  C,  Verification  and  Validation  Plan  for  the  ETE  Test  and  the  Phase  2 
Verification  and  Validation  Plan  for  the  End-to-End  Test. 

1.2  Verification  and  Validation  Tasks 

The  V&V  tasks  that  were  performed  during  the  ETE  risk  reduction  test  that  occurred  7-13  July 
1998  are  described  in  the  Phase  2  Verification  and  Validation  Plan  for  the  End-to-End  Test. 

1.3  V&V  Process  Models 

Within  this  report  reference  is  made  to  steps  enumerated  within  the  Distributed  Interactive 
Simulation  (DIS)  Nine  Step  Process  Model.  This  model  is  shown  below  as  Figure  1. 


Figure  1.  DIS  Nine  Step  VV&A  Process  Model 

The  process  model  and  its  accompanying  Recommended  Practice  for  Distributed  Interactive 
Simulation  —  Verification,  Validation,  and  Accreditation  (Draft-4  November  1996)  form  the 
basis  for  the  verification,  validation  and  accreditation  (W&A)  of  the  ETE  Test  synthetic 
environment  (SE). 
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The  DIS  Nine  Step  Process  Model  was  developed  with  a  conventional,  short-lived,  DIS  exercise 
in  mind,  as  opposed  to  a  test  of  a  major  system,  and  presupposes  a  full  complement  of  funds  and 
personnel  available  at  the  beginning  of  the  exercise  development.  This  disparity  was  brought  to 
the  attention  of  the  developers  of  the  DIS  Nine  Step  Process  Model,  and  the  conclusion  was 
reached  that  since  the  model  was  recommended  and  intended  for  tailoring  to  the  needs  of  the 
user,  the  model  would  continue  to  be  tied  to  the  DIS  exercise  development  and  construction 
process  contained  within  Institute  of  Electrical  and  Electronics  Engineers  (IEEE)  Standard 
1278.3. 

If  one  tailors  the  DIS  Nine  Step  Process  Model  to  the  joint  test  process,  then  the  process  model 
would  appear  as  shown  in  Figure  2. 
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In  the  JADS  ETE  Test  Process  Model,  test  events,  which  consist  of  the  planning,  construction 
and  assembling  of  the  SE,  integration  and  testing  of  the  SE,  accreditation  of  the  SE,  and  conduct 
of  the  test  all  proceed  on  the  left  side  from  top  to  bottom.  The  V&V  events,  to  include 
documentation,  proceed  to  the  right  for  each  test  event. 

2.  Applicable  Documents 

2.1  Documents 

ETE  Test  Activity  Plan  Appendix  C:  Verification  and  Validation  Plan  for  the  End-To-End 
(ETE)  Test 

Phase  2  Verification  and  Validation  Plan  for  the  End-to-End  Test. 

Department  of  Defense  Verification,  Validation  and  Accreditation  (W&A)  Recommended 
Practices  Guide,  November  1996 

Recommended  Practice  for  Distributed  Interactive  Simulation  —  Verification,  Validation,  and 
Accreditation,  4  November  1996 

3.  Verification  and  Validation  Tools 

The  following  nonstandard  software  were  used  in  the  ETE  Test  Phase  2  V&V: 

JADS  Toolbox 
JADS  Logger 

U.S.  Army  Simulation,  Training  and  Instrumentation  Command  (STRICOM)  Logger 

4.  Verification  Tasks 

This  section  of  the  Phase  2  V  &  V  results  states  the  verification  requirements  and  describes  the 
results  of  performing  the  procedures  as  stated  in  the  Phase  2  Verification  and  Validation  Plan  for 
the  End-to-End  Test. 

4.1  Janus  V6882 

4.1.1  Verify  Simulation 

Verify  that  Janus  V6882  is  capable  of  simulating  at  least  5000  entities  with  at  least  twenty-five 
percent  moving  using  a  commercial  off-the-shelf  (COTS)  product  costing  less  then  $100,000. 
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4. 1.1.1  Procedure 


The  ETE  Test  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Analysis  Center  (TRAC), 
White  Sands  Missile  Range  (WSMR),  New  Mexico,  site  representative  performed  the  following 
procedures: 

a)  Using  the  Force  editor  contained  within  Janus,  verify  that  each  scenario  to  be  used  for  the 
ETE  Phase  2  operational  test  (OT)  contains  at  least  5000  entities. 

b)  While  the  scenario(s)  is  running,  verify  that  at  least  1250  entities  move  during  the  course  of 
the  scenario.  (Note:  Not  all  vignettes  used  during  the  ETE  Phase  2  OT  call  for  at  least  1250 
moving  entities) 

c)  Determine  the  cost  of  the  COTS  product  used  to  run  the  above  scenarios. 

4. 1.1. 2  Results 

a)  ETE  Test  event  logs  maintained  by  the  ETE  Test  TRAC-WSMR  site  representative  contain 
the  following  entries: 

•  Date:  7  Jul  98  Time:  1310Z  Vignette  1/Scenario  901/9897  entities 

•  Date:  8  Jul  98  Time:  1256Z  Vignette  1/Scenario  901/9897  entities 

•  Date:  9  Jul  98  Time:  1300Z  Vignette  2/Scenario  902/9757  entities 

•  Date:  10  Jul  98  Time:  1306Z  Vignette  3/Scenario  903/9904  entities 

•  Date:  13  Jul  98  Time:  1255Z  Vignette  3/Scenario  903/9904  entities 

b)  ETE  Test  event  logs  maintained  by  the  ETE  Test  TRAC-WSMR  site  representative  contain 
the  following  entries: 

•  Date:  7  Jul  98  Time:  2054Z  Vignette  1  Scenario  901  Logfile  analysis/1639  separate 
entities  moved/max  of  601  entities  moving  at  one  time. 

•  Date:  13  Jul  98  Time:  2028Z  Vignette  3  Scenario  903  Logfile  analysis/4591  separate 
entities  moved/max  of  3268  entities  moving  at  one  time. 

c)  Price  list  for  suite  of  equipment  required  to  run  Janus  as  the  Virtual  Surveillance  Target 
Attack  Radar  System  (VSTARS)  stimulator  is  contained  in  Appendix  A.  Total  estimate  for 
suite  is  $19,800. 

4.1.2  Verify  Issue  of  Protocol  Data  Units 

Verify  that  Janus  V6882  issues  distributed  interactive  simulation  (DIS)  2.0.4  entity  state  protocol 
data  units  (ESPDU)  that  conform  in  content  and  format  with  the  DIS  2.0.4  standards  as  amended 
by  JADS. 
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4. 1.2.1  Procedure 


a)  Test  Control  and  Analysis  Center  (TCAC)  personnel  monitored  the  ESPDUs  arriving  from  the 
Janus  computer  at  TRAC-WSMR  using  the  JADS  toolbox.  The  type  of  protocol  data  units 
(PDU)  issued  by  Janus  were  noted  in  the  TCAC  log. 

b)  Following  the  ETE  risk  reduction  test,  the  ESPDUs  broadcast  by  Janus  were  analyzed  to 
determine  that  the  content  of  the  ESPDUs  conforms  with  the  DIS  2.0.4  standards  as  amended 
by  JADS  ETE  Test. 

4. 1.2. 2  Results 


a)  ETE  Test  event  logs  maintained  by  TCAC  site  representative  contain  the  following  entries: 

•  Date:  7  Jul  98  Time:  1326Z:  Run  number  from  WSMR  -  901-01 

•  Date:  7  Jul  98  Time:  1326Z:  Start  loggers,  Start  Janus 

•  Date:  7  Jul  98  Time:  1330Z:  TCAC  sees  PDUs 

•  Date:  7  Jul  98  Time:  133 1Z:  Confirm  ESPDUs 

•  Date:  8  Jul  98  Time:  131 1Z:  Run  number  901  01  from  WSMR 

•  Date:  8  Jul  98  Time:  1315Z:  Janus  started  TCAC  sees  PDUs 

•  Date:  8  Jul  98  Time:  1316Z:  Confirm  ESPDUs 


b)  Following  the  ETE  risk  reduction  test,  the  log  files  of  the  ESPDUs  broadcast  by  Janus  were 
analyzed  and  found  to  conform  with  the  DIS  2.0.4  standards  as  amended  by  JADS  ETE  Test. 
Representative  analyses  are  contained  within  Appendix  B. 


The  DIS  standard  requires  that  for  an  ESPDU,  the  “...  timestamp  shall  be  specified  using  a  32-bit 
unsigned  integer  representing  units  of  time  passed  since  the  beginning  of  the  current  hour.  The 
least  significant  bit  shall  indicate  whether  the  timestamp  is  absolute  or  relative.”  Because  the  ETE 
test  vignettes  used  in  Janus  involve  nearly  ten  thousand  entities  and  run  for  nearly  seven  hours,  it 
was  necessary  to  modify  the  standard  and  represent  time  as  a  32-bit  unsigned  integer  representing 
milliseconds  since  the  beginning  of  the  vignette  (relative  timestamp).  This  allows  the  tester  to 
trace  an  ESPDU  back  to  a  discrete  event  that  occurred  within  the  Janus  vignette.  This  is  a 
common  practice  in  most  DIS  synthetic  environments  (SE)  that  exist  for  periods  greater  than  one 
hour. 


4.1.3  Verify  Receipt  and  Action  of  Protocol  Data  Units 

Verify  that  Janus  V6882  will  receive  and  act  upon  each  ESPDU,  fire  PDU,  and  detonate  PDU 
issued  to  it  by  the  network  and  will  take  the  appropriate  action  as  dictated  by  the  PDU,  provided 
the  appropriate  data  sets  regarding  the  PDU  have  been  entered  into  the  simulation. 

4. 1.3.1  Procedures 


The  ETE  Test  TRAC-WSMR  site  representative  performed  the  following  procedures: 
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a)  Verify,  by  observing  the  Janus  monitor,  that  ESPDUs  received  from  other  nodes  within  the 
SE  are  displayed  correctly,  provided  they  are  located  within  the  Janus  scenario  area.  The  Fort 
Sill,  Oklahoma,  site  representative  provided  a  list  of  entities  represented  within  the  Tactical 
Army  Fire  Support  Model  (TAFSM)  and  their  locations  to  the  TCAC  for  dissemination  to  the 
other  nodes  within  the  SE. 

b)  Verify,  by  observing  the  Janus  monitor,  that  fire  and  detonate  PDUs  are  correctly  handled  by 
Janus.  Fort  Sill  site  representatives  notified  the  Janus  site  representative  of  the  predicted 
impact  point  before  each  Army  Tactical  Missile  System  (ATACMS)  was  fired.  In  addition, 
they  notified  the  Janus  site  representative  when  the  shot  was  fired  (fire  PDU)  and  when  it 
detonated  (detonate  PDU).  The  Janus  site  representative  made  an  entry  in  the  log  file  when 
the  detonate  PDU  was  received,  as  well  as  a  numbered  count  for  each,  and  also  recorded 
additional  information  describing  the  ‘killed’  entities. 

4. 1.3. 2  Results 

a)  Appendix  C:  Section  Cl  contains  the  results  of  observing  the  Janus  monitor.  Janus  correctly 
identified  all  Fort  Sill  entities  as  being  outside  the  scenario  terrain  box.  Appendix  C:  Section 
C2  contains  the  location  of  all  Fort  Sill  entities  as  supplied  by  TAFSM. 

b)  Appendix  C:  Section  C3  contains  the  ETE  Test  event  logs  for  TRAC-WSMR.  These  logs 
show  the  results  of  the  detonate  PDUs  transmitted  by  Fort  Sill.  Fire  PDUs  are  not  recorded 
as  the  entities  firing  are  outside  the  scenario  terrain  box  and  Janus  does  not  track  them. 
Additional  information  regarding  the  ‘killed’  entities  is  also  contained  within  the  ETE  Test 
event  logs. 

4.1.4  Verify  Acceptance  and  Processing  of  Scenarios 

Verify  that  the  simulation  will  accept  and  process  scenarios  with  a  duration  longer  than  eight 

hours. 

4. 1.4.1  Procedures 

The  Janus  site  representative  noted  in  the  site  log  file  the  duration  of  each  scenario  during  the 

ETE  risk  reduction  test. 

4. 1.4. 2  Results 

ETE  test  event  logs  maintained  by  the  ETE  Test  TRAC-WSMR  site  representative  contain  the 

following  entries: 

•  Date:  7  Jul  98  Time:  2032Z  Vignette  1/Scenario  901  Janus  game  time  08:00:46 

•  Date  8  Jul  98  Time:  2006Z  Vignette  1/  Scenario  901  Janus  froze  up  -  game  time  7:17:25 
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•  Date:  9  Jul  98  Time:  1337Z  Vignette  2/Scenario  902  Stop  Janus  -  game  time  1:21 
(Advanced  Field  Artillery  Tactical  Data  System  [AFATDS]  at  Fort  Hood  did  not  have 
ATACMS  launcher  data  for  Scenario  902) 

•  Date:  9  Jul  98  Time:  2002Z  Vignette  1/Scenario  901  Stop  Janus  -  game  time  7:10:40 

•  Date:  10  Jul  98  Time:  2020Z  Vignette  3/Scenario  903  Stop  Janus  -  game  time  7:00:30 
(Scenario  903  only  seven  hour  scenario) 

•  Date:  13  Jul  98  Time:  2010Z  Vignette  3/Scenario  903  Stop  Janus  -  game  time  7:00 
(Scenario  903  only  seven  hour  scenario) 

4.1.5  Verify  Interaction  With  Scenario 

Verify  that  the  operator  will  be  capable  of  fully  interacting  with  a  scenario  while  it  is  running. 

4. 1.5.1  Procedure 

The  Janus  site  representative  interacted  with  the  scenario  after  several  ATACMS  impacts  by 
altering  the  behavior  of  the  surviving  entities  within  the  immediate  area  of  the  impact.  A  log  entry 
was  made  after  each  interaction  indicating  the  results  of  the  interaction. 

4. 1.5. 2  Results 

ATACMS  missiles  were  only  fired  on  the  last  three  test  days  (070998,071098,071398)  and 
difficulty  was  experienced  the  first  day  in  determining  target  locations.  As  a  result,  no 
displacements  of  enemy  forces  were  performed  as  a  result  of  ATACMS  missions  fired  on  070998. 
ETE  Test  event  logs  maintained  by  the  ETE  Test  TRAC-WSMR  site  representative  contain  the 
following  entries: 

•  Date:  10  Jul  98  Time:  1640Z  Vignette  3/Scenario  903  Finished  interactive  move  of  31 
entities  from  5th  Corps  Headquarters  (HQ) 

•  Date:  10  Jul  98  Time:  1838Z  Vignette  3/Scenario  903  Finished  interactive  move  of  504th 
Supply  and  Transportation  Battalion  and  407th  Air  Defense  Artillery  Brigade  HQ 

•  Date:  13  Jul  98  Time:  1540Z  Vignette  3/Scenario  903  Finished  interactive  move  of 
vehicles  around  road  intersection  that  was  site  of  detonation 

4.1.6  Verify  Simulation  Capability  to  Proceed  in  a  Steplike  Fashion 

Verify  that  the  simulation  will  be  capable,  when  running,  of  proceeding  in  a  steplike  fashion  at  a 
pace  representative  of  near  real  time.  That  is,  when  a  unit  of  game  time  is  represented  within 
Janus  a  corresponding  amount  of  time  will  have  elapsed  since  the  start  of  the  simulation.  This 
feature  will  be  adjusted  by  tuning  the  scenario. 
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4. 1.6.1  Procedure 


The  Janus  site  representative  observed  the  clock  time  when  Janus  started  and  noted  it  in  the  site 
log.  The  elapsed  time  since  the  start  of  Janus  was  observed  periodically  (approximately  every  30 
minutes  and  compared  to  the  elapsed  game  time.  These  times  were  noted  in  the  site  log. 

4. 1.6.2  Results 

ETE  Test  event  logs  maintained  by  the  ETE  Test  TRAC-WSMR  site  representative  contain  the 
following  representative  entries: 

•  Date:  7  Jul  98  Time:  133 1Z  Vignette  1/Scenario  901  Start  Janus  DIS  at 
59:59/13:30:50Z  (Offset  equal  to  12:30:51) 

•  Date:  7  Jul  98  Time:  1602Z  Vignette  1/Scenario  901  Janus  3:31/  16:01 :47Z  (Offset 
equal  to  12:30:47) 

•  Date:  7  Jul  98  Time:  1934Z  Vignette  1/Scenario  901  Janus  06:30/  19:00:47Z  (Offset 
equal  to  12:30:47) 

•  Date:  13  Jul  98  Time:  1310Z  Vignette  3/Scenario  903  Start  Janus  0:00/13:10:00Z 
(Offset  equal  to  13:10:00) 

•  Date:  13  Jul  98  Time:  162 1Z  Vignette  3/Scenario  903  Janus  3:10:59/16:21 :04Z  (Offset 
equal  to  13:10:05) 

•  Date:  13  Jul  98  Time:  1840Z  Vignette  3/Scenario  903  Janus  5:30:00/18:40:04Z  (Offset 
equal  to  13:10:04) 

4.1.7  Verify  Capability  of  Using  Scenarios 

Verify  that  the  simulation  will  be  capable  of  utilizing  scenarios  conducted  upon  terrain 
representing  a  simulation  area  of  at  least  200  kilometers  (km)  by  200  km. 

4. 1.7.1  Procedure 

Prior  to  the  start  of  each  scenario,  the  Janus  site  representative  determined  the  coordinates  of 
each  comer  of  the  Janus  terrain  box  and  noted  them  in  the  site  log. 

4. 1.7.2  Results 

ETE  Test  event  logs  maintained  by  the  ETE  Test  TRAC-WSMR  site  representative  contain  the 
following  entries: 

a)  Date:  7  Jul  98  Time:  131 5Z  Vignette  1/Scenario  901  Terrain: 

•  Upper  Left  3 1°  32’  34"  N  45°  28'  39"  E 

•  Lower  Left  30°  00'  47"  N  45°  28'  12"  E 

•  Upper  Right  31°  31'  25"  N  47°  15'  41"  E 

•  Lower  Right  29°  59'  42"  N  47°  13'  34"  E 
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Result  is  a  terrain  box  170  km  by  170  km 

b)  Date:  10  Jul  98  Time:  1309Z  Vignette  3/Scenario  903  Terrain: 

•  Upper  Left  3 1°  48'  54"  N  45°  28'  38"  E 

•  Lower  Left  30°  16'  56"  N  45°  18'  10"  E 

•  Upper  Right  31°  47'  45"  N  47°  16’  12"  E 

•  Lower  Right  30°  15'  5 1"  N  47°  14'  02"  E 

Result  is  a  terrain  box  170  km  by  170  km 

4.2  Tactical  Army  Fire  Support  Model 

4.2.1  Verify  Acceptance  of  Artillery  Missions 

Verify  that  TAFSM  will  accept  artillery  missions  using  tactical  fire  (TACFIRE)  or  Advanced  Field 
Artillery  Tactical  Data  System  (AFATDS)  messages  encapsulated  in  a  DIS  PDU.  Simulation  will 
issue  a  fire  and  detonate  PDU  whenever  an  ATACMS  missile  is  fired  and  subsequently  detonated. 

4. 2. 1.1  Procedure 

The  Fort  Sill  site  representative  noted  in  the  site  log  when  each  artillery  mission  was  received  by 
TAFSM  via  a  DIS  PDU.  The  site  representative  subsequently  noted  when  the  fire  and  detonate 
PDUs  corresponding  to  the  artillery  mission  were  issued.  This  information  was  provided  by  a 
post-test  history  file  and  observer  log  sheets. 

4. 2. 1.2  Results 

During  the  first  functionality  and  integration  test,  the  decision  was  made  by  the  ETE  Test 
manager  to  modify  the  design  of  the  ETE  Test  SE.  The  modification  consisted  of  having  the 
AFATDS  located  at  Fort  Hood  communicate  directly  with  the  AFATDS  at  Fort  Sill.  This 
communication  would  be  accomplished  using  standard  AFATDS  message  traffic  instead  of  DIS 
PDUs.  The  AFATDS  at  Fort  Sill  would  then  communicate  with  a  network  interface  unit,  called  a 
personal  computer  network  interface  unit  (PCNIU),  which  would  pass  the  AFATDS  traffic  to 
TAFSM  using  signal  and  transmitter  PDUs.  After  TAFSM  processes  the  fire  mission  sent  to  it  by 
AFATDS,  TAFSM  then  broadcasts  a  fire  PDU  when  the  ATACMS  missile  is  launched  and  a 
detonate  PDU  when  the  missile  impacts. 

The  DIS  logger  located  at  Fort  Sill  only  records  DIS  PDUs.  AFATDS  traffic  is  recorded  by  the 
AFATDS  devices  at  each  location.  An  example  of  AFATDS  traffic  is  included  as  Appendix  D: 
Section  D 1 . 

A  record  of  the  DIS  PDUs  generated  by  Fort  Sill  is  included  as  Appendix  D:  Section  D2.  As  can 
be  seen,  signal,  transmitter,  fire,  and  detonate  PDUs  are  generated  by  Fort  Hood. 
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Finally,  the  site  event  logs  are  included  as  Appendix  D:  Section  D3. 

4.3  End-To-End  Test  Synthetic  Environment 

4.3.1  Perform  Compliance  Standards  Verification  (Step  2) 

Compliance  standards  verification  will  be  performed  after  the  ETE  Risk  Reduction  Test  and  will 
consist  of  an  inspection  of  a  sampling  of  PDUs  emitted  by  each  simulation  to  insure  proper 
adherence  to  the  prescribed  format.  In  addition,  a  model  interface  analysis  of  the  SE  will  be 
conducted  to  determine  if  the  submodels  within  the  SE  are  correctly  communicating  using  DIS 
PDUs. 

43.1.1  Procedures 

a)  Following  the  ETE  risk  reduction  test,  the  TCAC  PDU  log  file  for  each  day  was  randomly 
sampled  to  ensure  proper  adherence  to  the  prescribed  format.  Sample  size  was  at  least  100 
PDUs,  where  possible,  from  each  simulation  generating  PDUs. 

b)  Within  the  ETE  Test  SE  as  developed  for  the  OT,  there  are  two  instances  of  submodels 
communicating  using  DIS  PDUs.  These  instances  are  the  transfer  of  ESPDUs  from  Janus  to 
the  Virtual  Surveillance  Target  Attack  Radar  System  (VSTARS)  and  the  transfer  of  fire  and 
detonate  PDUs  from  TAFSM  to  Janus.  The  PDU  log  files  taken  at  the  above  nodes  were 
compared  post  test  to  determine  how  reliably  the  simulations  communicated 
amongthemselves. 

43.1.2  Results 

a)  The  PDU  log  files  for  each  test  day  were  analyzed  using  the  JADS  Toolbox.  Data  extracted 
from  the  log  files  verified  that  the  simulations  broadcasting  PDUs  within  the  ETE  Test  SE 
adhered  to  the  prescribed  format.  Appendix  B  contains  a  representative  analysis  file  of 
ESPDUs  broadcast  by  Janus  at  TRAC-WSMR  on  13  July  1998. 

b)  An  analysis  of  how  well  two  simulations  communicate  between  themselves  involves  two 
states.  Either  the  network  connecting  the  simulations  is  in  a  normal  state  and  communication 
occurs,  or  the  network  is  down  and  no  communication  occurs.  This  verification  task  will 
consider  only  the  normal  state  of  the  network,  when  communication  occurs.  Network 
outages,  when  no  communication  occurs,  will  be  discussed  at  a  later  verification  step. 

During  the  five  days  of  testing,  more  than  a  million  and  a  half  ESPDUs  were  broadcast  from  Janus 
and  received  by  VSTARS.  In  an  attempt  to  reduce  these  data  to  a  more  manageable  amount, 
each  test  day  was  divided  into  one-hour  periods  and  then  samples  of  16000  contiguous  ESPDUs 
were  selected  on  a  random  basis.  These  samples  were  then  compared  between  the  two  sites  and 
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the  number  of  PDUs  dropped  during  normal  network  operation  was  observed.  Appendix  E 
contains  a  portion  of  one  of  the  comparison  files  with  the  results  shown. 

Overall,  out  of  96000  PDUs  compared,  only  30  were  lost  in  transmission,  which  would  indicate 
that  the  communication  between  Janus  and  VSTARS  is  highly  reliable  when  the  network  is 
operating  normally.  Further  analysis  of  overall  network  performance  will  be  discussed  later  in  this 
report. 

The  transfer  of  fire  and  detonate  PDUs  between  Fort  Sill  and  TRAC-WSMR  was  without  error 
during  the  three  days  of  testing  when  ATACMS  missiles  were  fired.  Appendix  E:  Section  E3 
contains  the  comparison  file  for  13  July  1998. 

4.3.2  Perforin  Capability  Verification  (Step  6) 

The  compatibility  verification  will  be  performed  by  the  ETE  tTest  V&V  team  during  Phase  2  of 
the  ETE  Test.  The  compatibility  verification  will  complete  the  verification  of  the  ETE  Test  SE  by 
ensuring 

a)  modeling  and  simulation  (M&S)  components  exchange  data  and  interact  appropriately  with 
each  other; 

b)  individual  components  correctly  use  the  common  data  (e.g.,  terrain,  weather)  to  generate  their 
portion  of  the  synthetic  environment;  and 

c)  the  overall  implementation  is  adequate  to  address  the  exercise  requirements. 

This  activity  involves  five  major  tasks:  Evaluate  design  versus  implementation,  evaluate 
compatibility,  evaluate  interface  implementation,  assess  instrumentation  requirements,  and 
evaluate  impact  of  operator  proficiency. 

Evaluate  Design  Versus  Implementation:  This  task  will  determine  if  the  design  is  sufficient  to 
ensure  the  adequacy  of  the  overall  implementation  by  comparing  the  design  as  documented  (e.g., 
conceptual  model,  component  compliance  profiles  and  fidelity  characterizations)  and  the  exercise 
configuration.  The  V&V  team  will  participate  in  an  exercise  development  walk-through  and 
apply  a  series  of  checks  to  compare  the  physical  configuration  to  the  documented  design.  In 
addition,  functional  testing  will  be  applied  to  assess  performance  of  the  SE  over  the  course  of  the 
test. 

Evaluate  Compatibility:  This  task  will  determine  if  the  individual  components 

a)  represent  system  performance  as  required  for  the  exercise; 

b)  transfer  information  to  and  from  the  network  without  corruption; 

c)  share  common  perspectives  of  the  virtual  reality  produced  by  the  exercise;  and 

d)  employ  database  elements,  shared  models  and  support  systems  appropriately. 

Evaluate  Interface  Implementation:  This  task  focuses  on  network  performance  needs, 
interface  implementation  issues,  and  identification  of  changes  in  the  exercise  configuration  that 
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could  impact  operation  of  the  network.  The  V&V  team  will  inspect  the  hardware  configuration 
and  review  data  collection  and  transfer  (e.g.,  PDUs)  among  components  to  determine  if  the 
interface  implementation  is  in  accordance  with  interface  specifications.  The  V&V  team  will  also 
evaluate  the  results  of  network  loading  and  latency  tests  for  possible  impacts  on  simulation 
results. 

Assess  Instrumentation  Requirements:  The  V&V  team  will  evaluate  the  adequacy  of  the 
instrumentation  requirements  for  V&V  purposes. 

Evaluate  Impact  of  Operator  Proficiency:  The  V&V  team,  in  association  with  identified 
subject  matter  experts  (SMEs),  will  observe  and  evaluate  the  performance  of  operators  to 
determine  if  they  possess  the  appropriate  skill  level  to  perform  the  functions  required  for  the  test. 

4.3.2. 1  Procedures 

a)  Evaluate  Design  Versus  Implementation:  This  task  has  been  performed  over  the  course  of 
the  last  four  functionality  and  integration  tests.  Following  the  ETE  risk  reduction  test  the 
V&V  team  reviewed  the  overall  implementation  of  the  test  and  compared  it  to  the 
documented  design.  Both  compliance  with  the  documented  design  and  functionality  were 
addressed.  The  report  on  this  task  will  address  not  only  deviations  from  the  documented 
design  but  also  the  impact  of  the  deviation  upon  the  functionality  of  the  SE,  if  any. 

b)  Evaluate  Compatibility:  This  task  has  been  performed  over  the  course  of  the  last  four 
functionality  and  integration  tests.  Following  the  ETE  risk  reduction  test  the  V&V  team 
evaluated  the  compatibility  of  the  individual  components.  The  report  on  this  task  addresses 
areas  of  incompatibility,  their  impact  upon  the  SE,  and  projected  fixes,  if  any. 

c)  Evaluate  Interface  Implementation:  This  task  was  performed  by  the  V&V  team  following 
the  ETE  risk  reduction  test  using  data  collected  during  the  test.  The  report  on  this  task 
addresses  performance  and  possible  impacts  on  simulation  results. 

d)  Assess  Instrumentation  Requirements:  This  task  was  performed  by  the  V&V  team 
following  the  ETE  risk  reduction  test.  It  consisted  of  a  review  of  data  collected  and 
instrumentation  present  during  the  test.  The  report  on  this  task  addressed  V&V  areas  that  are 
impacted  by  lack  of  instrumentation  and  the  extent  of  the  impact  upon  V&V. 

e)  Evaluate  Impact  of  Operator  Proficiency:  For  the  system  under  test  (the  Joint  Surveillance 
Target  Attack  Radar  System  [Joint  STARS])  this  V&V  task  is  impacted  by  the  fact  that  the 
operators  are  generally  the  SMEs.  The  operators  can  be  divided  into  two  groups,  those 
associated  with  the  E-8C  and  those  associated  with  the  light  ground  support  module  (LGSM). 
All  E-8C  operators  are  required  to  be  certified  annually  as  to  their  proficiency  in  performing 
their  assigned  tasks.  All  operators  used  in  the  risk  reduction  test  were  evaluated  during  the 
conduct  of  the  test.  The  report  on  this  task  addresses  operator  efficiency  and  its  impact  upon 
the  OT. 
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4. 3.2.2  Results 


a)  Evaluate  Design  Versus  Implementation:  As  mention  previously,  during  the  first 
functionality  and  integration  test  the  decision  was  made  by  the  ETE  Test  manager  to  modify 
the  design  of  the  ETE  Test  SE.  The  modification  consisted  of  having  the  AFATDS  located  at 
Fort  Hood  communicate  directly  with  the  AFATDS  at  Fort  Sill.  This  communication  would 
be  accomplished  using  standard  AFATDS  message  traffic  instead  of  DIS  PDUs.  The 
AFATDS  at  Fort  Sill  would  then  communicate  with  a  network  interface  unit,  called  a  PCNIU, 
which  would  pass  the  AFATDS  traffic  to  TAFSM  using  signal  and  transmitter  PDUs.  After 
TAFSM  processes  the  fire  mission  sent  to  it  by  AFATDS,  TAFSM  then  broadcasts  a  fire 
PDU  when  the  ATACMS  missile  is  launched  and  a  detonate  PDU  when  the  missile  impacts. 

The  remainder  of  the  SE  remains  as  developed  during  the  detailed  design  and  described  within  the 
ETE  Test  Activity  Plan  Appendix  C:  Verification  and  Validation  Plan  for  the  End-To-End 
(ETE)  Test.  Basically,  the  remainder  of  the  SE  consists  of  a  scenario  generator  or  wargame, 
Janus;  an  E-8C  simulation,  VSTARS;  an  actual  LGSM  at  Fort  Hood,  which  receives  radar 
reports  from  VSTARS;  and  a  target  analysis  cell  (TAC)  that  issues  ATACMS  fire  missions 
through  AFATDS.  The  TAC  is  a  subset  of  the  corps  analysis  control  element  (ACE).  Once 
TAFSM  issues  the  fire  and  detonate  PDUs  described  above,  Janus  calculates  the  effects  of  the 
missile  shot  and  the  end-to-end  loop  begins  again  with  Janus  transmitting  the  results  to 
VSTARS. 

The  deviation  from  the  documented  design  described  above  was  in  actuality  a  simplification  of  the 
design.  The  changing  of  an  AFATDS  message  generated  at  Fort  Hood  into  a  DIS  PDU  for 
transmission  to  another  AFATDS  located  at  Fort  Sill,  with  the  resultant  changing  of  the  PDU 
back  to  the  AFATDS  message,  was  a  needless  complication  and  a  potential  source  of  error 
within  the  SE.  Use  of  the  system’s  means  of  communication  also  added  validity  to  the  SE,  as 
any  problems  or  errors  experienced  were  real  errors  as  opposed  to  errors  caused  by  advanced 
distributed  simulation  (ADS). 

The  functionality  of  the  SE  design  is  documented  by  observer  logs  and  analysis  of  simulation  and 
PDU  log  files,  as  previously  discussed.  The  true  functionality  of  the  SE  is  best  described, 
however,  by  a  statement  made  by  one  of  the  members  of  the  TAC.  He  stated  that  this  was  the 
first  time  to  his  knowledge  that  the  TAC  was  able  to  perform  its  assigned  mission  under 
tactical  conditions  in  a  doctrinally  correct  manner  with  all  systems  performing  as  designed. 

b)  Evaluate  Compatibility:  All  ADS  elements  of  the  SE  were  compatible  with  each  other  to 
the  degree  required.  This  is  not  a  surprising  result  as  all  elements  were  either  designed  and 
built  to  be  compatible  or  were  modified  to  be  compatible  over  the  course  of  the  conduct  of 
four  functionality  and  integration  tests. 

This  was  not  the  case  with  the  fielded  real  systems  used  within  the  SE.  There  were  many 
incompatibility  problems  among  these  fielded  systems  that  persisted  well  into  the  ETE  risk 
reduction  test.  As  stated  earlier,  the  TAC  had  never  had  the  opportunity  to  perform  this 
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function,  under  tactical  conditions  in  a  doctrinally  correct  manner  prior  to  the  ETE  risk 
reduction  test. 

c)  Evaluate  Interface  Implementation:  This  portion  of  the  V&V  considers  the  technical 
performance  of  the  SE  and  the  interfaces  and  networking  equipment  involved  in  supporting 
the  SE. 

Appendix  F  contains  the  ETE  Test  risk  reduction  1  analysis  summary.  In  summary,  the  SE  was 
fully  available  96.49%  of  the  time  for  testing.  During  the  period  (2  hours  37  minutes)  the 
JADS  ground  data  terminal  1553  bus  interface  unit  was  down,  testing  could  still  be  conducted 
at  the  VSTARS  node.  This  left  only  two  minutes,  out  of  a  total  of  35  hours  7  minutes,  that 
the  SE  was  unavailable  for  testing  of  any  kind. 

d)  Assess  Instrumentation  Requirements:  No  additional  instrumentation  requirements  for  the 
ETE  Test  were  identified  during  the  ETE  risk  reduction  test.  All  technical  measures  could  be 
evaluated  using  data  collected  with  existing  instrumentation.  Operational  measures  were 
evaluated  using  observer  logs  combined  with  simulation  or  system  history  tapes. 

This  is  not  to  say  that  additional  instrumentation  could  not  be  added  to  support  a  different  test 
design.  As  an  example,  the  unclassified  portion  of  the  ETE  Test  network  is  not  instrumented 
with  Spectrum™  to  measure  bandwidth  utilization,  since  it  must  be  by  design  equal  to  or  less 
then  the  bandwidth  utilization  on  the  classified  side.  If  used  for  a  different  test  design,  this 
may  not  be  the  case  and  the  unclassified  portion  of  the  ETE  Test  network  would  need  to  be 
instrumented  for  the  measurement  of  bandwidth  utilization.  This  point  is  brought  out  to 
emphasize  that  this  V&V  directly  applies  only  to  the  ETE  Test  SE. 

e)  Evaluate  Impact  of  Operator  Proficiency:  The  impact  of  operator  proficiency  is  very  high 
for  the  ETE  Test.  Even  though  all  operators  are  ‘trained’  in  the  operation  of  their  systems, 
both  the  Army  and  Air  Force  have  acknowledged  that  their  existing  training  systems  leave  a 
lot  to  be  desired.  In  fact,  both  services  have  expressed  a  desire  to  utilize  VSTARS  in  future 
training  systems. 

In  addition  to  the  absence  of  effective  training  systems,  peacetime  training  missions  do  not  expose 
the  operators  to  many  of  the  tasks  and  functions  they  are  expected  to  perform  during 
hostilities.  As  a  result,  during  the  conduct  of  the  modified  Turing  test  one  of  the  test  subjects, 
a  ‘trained  operator,'  remarked  that  during  the  test  he  had  the  opportunity  to  perform  a  certain 
function  for  the  first  time  in  the  three  years  he  had  been  assigned  to  the  system.  Additionally, 
during  the  modified  Turing  test  another  operator  asked  for  a  Joint  STARS  Operators  Manual, 
because  he  was  asked  to  do  things  that  he  did  not  normally  have  the  opportunity  to  do.  With 
the  manual  he  was  able  to  successfully  accomplish  the  task. 

The  end  result  of  all  of  this  is  that  even  though  no  additional  training  is  required  to  use  the  SE,  the 
operators  must  use  the  SE  to  leam  or  practice  the  tasks  they  are  expected  to  perform  using 
the  real  system.  This  is  a  result  of  the  fact  that  to  date  only  the  SE  and  war  places  the  system 
and  the  operator  in  an  operationally  realistic  environment. 
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As  a  result,  the  functionality  and  integration  tests  and  the  risk  reduction  test  have  been  not  only 
tests  and  rehearsals  but  also  training  sessions  for  the  operators.  Are  they  trained  sufficiently 
well  so  as  to  participate  effectively  in  the  test?  A  difficult  question  to  answer,  as  there  is  no 
way  to  test  them  without  using  the  SE.  At  best,  we  can  only  say  that  they  are  better  trained 
then  when  we  started  and  they  can  successfully  execute  the  requirements  of  the  test  using  the 
SE. 

5.  Validation  Tasks 

5.1  Janus  V6882 

5.1.1  Validate  That  Janus  Represents  Vehicle  Behavior 

Validate  that  Janus  V6882  represents  vehicle  behavior  to  the  degree  detectable  by  the  Joint 

STARS.  This  capability  will  be  judged  based  upon  viewing  vehicle  movement  upon  the  Joint 

STARS  operator  workstation.  Joint  STARS  operator  SMEs  will  be  used  to  evaluate  these 

criteria. 

5. 1.1.1  Procedures 

a)  During  the  ETE  risk  reduction  test,  operators  at  both  Northrop  Grumman,  Melbourne, 
Florida,  and  Fort  Hood,  Texas,  were  queried  as  to  whether  or  not  the  vehicle  movement  seen 
on  the  workstation  monitors  appeared  realistic.  If  it  was  considered  unrealistic,  the  operators 
were  asked  what  impact  this  had  on  the  performance  of  their  assigned  tasks.  Their  responses 
became  a  part  of  the  site  log  file  and  were  reviewed  post  test  by  the  ETE  Test  V&V  team. 

b)  In  addition,  the  TRAC-WSMR  PDU  log  files  were  analyzed  post  test  and  reviewed  to  ensure 
that  Janus  V6882  properly  represents  vehicle  movement. 

5. 1.1. 2  Results 

a)  Observation  of  workstation  monitors  by  members  of  the  V&V  team  and  reports  from 
operators  during  the  risk  reduction  test  revealed  instances  of  unrealistic  vehicle  movement 
under  certain  scenario  conditions.  The  anomaly  manifested  itself  by  having  portions  of 
convoys  missing  turns  and  wandering  off  into  the  desert.  The  lost  portions  of  the  convoys 
would  then  jump  back  into  formation  after  a  period  of  time  and  resume  normal  movement. 
Another  manifestation  of  the  anomaly  had  elements  of  a  convoy  miss  a  turn  and  proceed  to 
cross  the  Euphrates  River  without  benefit  of  a  bridge.  As  an  aside,  the  operator  automatically 
verified  that  a  bridge  did  not  exist  by  taking  a  virtual  synthetic  aperture  radar  (SAR)  image  of 
the  crossing  site.  Logs  of  the  observers  who  worked  with  the  operators  are  included  as 
Appendix  F. 
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Visual  analysis  of  the  moving  target  indicator  (MTI)  radar  reports  by  ETE  Test  V&V  team 
members  led  them  to  the  conclusion  that  VSTARS  was  not  receiving  change  of  state  ESPDUs 
from  the  Janus  simulation  located  at  TRAC-WSMR.  VSTARS  will  dead  reckon  an  entity 
based  on  the  last  PDU  received  until  a  new  PDU  is  received.  A  quick  check  with  the  TCAC 
revealed  that  the  network  was  functioning  normally  and  that  VSTARS  was  receiving  all  PDUs 
broadcast  by  Janus. 

Measurement  of  the  time  differential  between  when  the  entities  began  to  wander  off  and  when 
they  leaped  back  into  formation  led  the  ETE  Test  V&V  team  members  to  believe  that  the 
anomaly  was  related  to  the  heartbeat  period  used  for  the  scenario.  Further  observation 
revealed  that  the  anomaly  only  occurred  during  periods  of  heavy  scenario  activity.  As  an 
example,  at  the  time  the  anomaly  was  observed,  the  scenario  (Scenario  903)  contained  nearly 
10,000  entities  with  more  than  3000  of  the  entities  moving.  The  anomaly  was  not  observed 
when  simpler,  less  busy  scenarios  (Scenario  901)  were  run. 

For  clarification  it  is  necessary  to  state  that  Janus,  like  the  majority  of  DIS-compliant  simulations, 
broadcasts  two  types  of  ESPDUs.  Change  of  state  ESPDUs  are  a  result  of  an  entity  starting, 
stopping,  turning  or  changing  speed  beyond  preset  limits.  They  are  also  referred  to  as  dead¬ 
reckoning  PDUs  and  are  broadcast  whenever  the  entity  meets  the  change  of  state  condition. 
Heartbeat  ESPDUs  are  broadcast  at  a  preset  rate  for  all  entities  that  have  not  experienced  a 
change  of  state  during  the  heartbeat  period. 

As  an  example,  for  a  10000  entity  scenario  with  no  entities  moving  and  a  heartbeat  period  of  10 
minutes,  an  observer  would  see  10000  ESPDUs  broadcast  over  a  ten  minute  period  in  entity 
order.  Once  entities  started  moving,  however,  an  observer  would  expect  to  see  more  than 
10000  PDUs  broadcast  and  they  would  not  be  in  entity  order.  The  extra  PDUs  are  a  result  of 
a  change  in  state  after  the  heartbeat  PDU  for  that  entity  has  already  been  broadcast  or  multiple 
changes  of  state  during  the  10  minute  heartbeat  period. 

Obviously,  an  anomaly  of  this  type  will  have  a  major  impact  on  the  performance  of  the  operators' 
tasks.  Fortunately,  this  anomaly  occurs  only  under  certain  conditions  and  solutions  are 
potentially  available. 

b)  Post-test  analysis  of  the  ESPDU  log  files  revealed  that  what  the  ETE  Test  V&V  team 
suspected  was  true.  The  rate  at  which  heartbeat  ESPDUs  are  broadcast  and  the  heartbeat 
period  must  be  set  before  the  scenario  is  executed  in  Janus.  A  low  rate  is  desired  because  the 
information  contained  in  the  ESPDU  will  be  sent  to  the  E-8C  by  satellite  communication 
(SATCOM),  and  the  network  routers  start  to  drop  increasing  numbers  of  PDUs  as  the  rate 
increases.  The  heartbeat  period  is  set  based  on  the  rate  and  the  number  of  entities  within  the 
scenario.  Previous  practice  was  to  set  the  heartbeat  period  very  close  to  the  minimum 
required  for  the  rate  used  in  the  simulation  run. 

This  worked  fine  until  more  than  entities  started  moving  all  over  the  landscape.  At  this  point, 

Janus  simply  could  not  do  everything  in  the  time  allocated  and  failed  to  send  a  PDU  for  each 

entity  within  the  scenario  for  a  specified  heartbeat  period.  Instead,  it  would  start  the  next 
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heartbeat  period  and,  depending  upon  the  load  during  that  period,  correct  the  location  of  the  lost 
entities  or  fall  even  farther  behind.  For  those  entities  that  were  stationary,  this  made  no 
difference.  The  omitted  change  of  state  PDUs  for  entities  turning,  however,  resulted  in  the 
entities  wandering  off  into  the  desert. 

5. 1.1. 3  Solutions 

There  are  three  immediately  available  solutions  to  the  problem  just  described  that  do  not  require 
any  modifications  to  Janus.  Unfortunately,  the  problem  occurred  and  was  identified  during  the 
last  two  hours  of  the  last  day  of  the  risk  reduction  test.  A  full  check-out  of  any  solution  will  have 
to  wait  until  the  SE  is  reestablished  during  the  rehearsal  and  test  preparation  period  prior  to  the 
ETE  operational  test.  The  possible  solutions  are 

a)  Increase  the  rate  at  which  Janus  issues  heartbeat  PDUs  to  the  maximum  allowed  by  network 
constraints  and  increase  the  heartbeat  period  to  the  maximum  allowed.  This  will  allow  Janus 
to  issue  PDUs  at  a  faster  rate  and  will  allow  Janus  more  time  to  accomplish  required  PDU 
activities.  This  will  have  no  detrimental  effect  upon  the  test  since  the  heartbeat  is  used  as  a 
backup  in  case  a  change  of  state  PDU  is  lost  in  transmission.  Increasing  the  heartbeat  period 
would  mean  that  it  could  take  up  to  fifteen  minutes  to  replace  the  lost  PDU  as  opposed  to 
eleven  minutes.  Also,  as  discussed  in  Section  4.3.1.2.b),  only  30  PDUs  were  lost  out  of 
96000  during  normal  network  operation.  None  of  the  PDUs  lost  were  change  of  state  PDUs 
that  required  replacement. 

b)  Prior  to  the  beginning  of  entity  movement,  turn  off  the  heartbeat.  This  will  result  in  Janus 
issuing  only  change  of  state  PDUs  as  the  changes  occur.  This  feature  was  built  into  Janus  as  a 
contingency  in  case  the  simulation  experienced  a  problem  with  processor  overload.  As  just 
stated,  the  heartbeat  is  needed  within  the  SE  only  as  a  backup  in  case  of  lost  PDUs.  VSTARS 
will  continue  too  dead  reckon  on  the  last  PDU  received  until  a  new  one  is  received. 
Furthermore,  the  rate  of  PDU  loss  is  dependent  upon  the  PDU  transmission  rate.  Lowering 
the  PDU  rate  by  turning  off  the  heartbeat  will  lower  the  loss  rate  to  the  point  that  any  errant 
entity  will  be  lost  in  the  noise. 

c)  If  the  previous  solutions  do  not  solve  the  problem,  it  is  always  possible  to  decrease  the 
amount  of  activity  within  a  scenario.  Initial  observations  have  established  that  there  appears 
to  be  a  threshold  of  activity  above  which  the  anomaly  occurs  and  realism  disappears.  If 
required,  this  threshold  will  be  quantified  and  the  scenarios  used  during  the  ETE  Test  will  be 
revised  to  stay  below  the  threshold.  Initial  indications  are  that  even  if  we  were  required  to 
remain  below  this  threshold,  there  would  still  be  many  more  entities  conducting  tactical 
movement  than  can  be  found  using  live  test  ranges. 

Once  the  problem  is  solved,  an  addendum  will  be  published  to  this  report  detailing  the  solution 
arrived  at  and  the  observations  of  the  operators  at  that  time. 

5.2  End-To-End  Test  Synthetic  Environment 
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5.2.1  Perform  Validation  (Step  7) 


The  validation  of  the  ETE  Phase  2  OT  SE  was  performed  by  the  ETE  Test  V&V  team,  assisted 
by  Northrop  Grumman  and  the  Joint  STARS  Joint  Test  Force,  during  the  ETE  risk  reduction  test. 

The  validation  of  the  ETE  Phase  2  OT  SE  was  intended  to  ensure  that  the  integrated  simulations 
are  adequate  to  satisfy  the  ETE  Test  requirements  such  that  the  SE  behavior  and  performance 
map  sufficiently  to  real-world  counterparts  for  the  specific  application;  performances  and 
representations  of  the  simulated  entities  are  sufficient  to  support  the  intended  application;  and 
testing  has  been  done  to  address  acceptance  criteria. 

This  activity  consists  of  five  basic  tasks:  Establish  context  for  validation  activities,  evaluate 
configuration  interoperability,  perform  effectiveness  evaluation,  evaluate  test  results,  and  evaluate 
operator  performance. 

5.2. LI  Procedures 

a)  Establish  Context  for  Validation  Activities:  This  validation  task  has  been  performed  in 
preparation  for  the  ETE  risk  reduction  test.  The  scope  of  the  validation  effort  is  specified  by 
the  Phase  2  Verification  and  Validation  Plan  for  the  End-to-End  Test.  No  new  acceptability 
criteria  have  been  identified  during  the  course  of  four  functionality  and  integration  tests,  and 
potential  shortcomings  and  limitations  of  the  SE  have  been  identified. 

b)  Evaluate  Configuration  Interoperability:  This  validation  task  was  performed  by  the  V&V 
team  following  the  ETE  risk  reduction  test.  It  consisted  of  verifying  the  mapping  of  the 
individual  components  of  the  SE  to  the  detailed  design  and  validating  that  the  individual 
components  performed  as  required  by  the  design.  PDU  log  files  and  observer  log  files  were 
used  to  conduct  this  validation  task. 

c)  Perform  Effectiveness  Evaluation:  This  validation  task  is  a  follow-on  to  the  previous  task. 
Once  it  was  ascertained  that  the  individual  components  performed  as  required,  their 
effectiveness  was  ascertained  by  tracing  exercise  performance  data  to  the  acceptability  criteria 
and  evaluating  the  data  for  accuracy,  sufficiency,  and  appropriateness.  PDU  log  files  and 
observer  log  files  were  used  to  conduct  this  validation  task. 

d)  Evaluate  Test  Results:  This  validation  task  is  also  a  follow-on  to  the  previous  task.  After  the 
effectiveness  of  the  individual  components  were  determined,  the  effectiveness  of  the  overall 
SE  was  evaluated  and  compared  to  the  real  world  represented  by  the  SE.  PDU  log  files  and 
observer  log  files  were  used  to  conduct  this  validation  task. 

e)  Evaluate  Operator  Performance:  This  validation  task  considered  operator  performance 
while  working  within  the  SE  vice  the  real  world.  Advanced  Technology  Work  Station 
(ATWS)  operators  and  LGSM  operators  were  queried  by  the  test  observers  after  each  test  run 
as  to  differences  in  individual  performance  between  the  two  environments.  Differences  and 
their  impact  upon  the  test  were  noted  in  the  observer  log. 
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5. 2. 1.2  Results 


a)  Establish  Context  for  Validation  Activities:  As  stated  above,  this  task  was  performed  prior 
to  the  risk  reduction  test.  During  the  conduct  of  the  risk  reduction  test,  no  new  acceptability 
criteria  were  identified.  If  required  to  implement  the  third  solution  to  the  Janus  problem, 
identified  in  5.1.1.3c),  this  would  place  a  limitation  upon  the  SE  as  previously  discussed. 

b)  Evaluate  Configuration  Interoperability:  The  configuration  interoperability  of  the  SE  has 
previously  been  discussed  within  this  report  as  a  part  of  the  verification  of  the  SE  and  its 
components.  Further  evidence  of  the  SE  configuration  interoperability  is  attached  as 
Appendix  G:  Section  Gl:  Risk  Reduction  Test  Quick-Look  Report  and  Appendix  G:  Section 
G2:  Operational  Measures  Report. 

c)  Perform  Effectiveness  Evaluation:  The  effectiveness  of  the  SE  has  also  been  discussed  as  a 
part  of  the  verification  of  the  SE  and  its  components.  It  is  apparent  that  the  SE  is  an  effective 
tool  in  evaluating  the  End-to-End  Test  concept  under  operational  conditions.  The  fact  that 
actual  operators  are  able  to  perform  operational  functions  for  the  first  time  using  the  SE  is 
further  proof  that  the  SE  is  an  effective  tool  for  operational  testing.  In  addition,  actual 
operational  measures  taken  from  the  Joint  STARS  multiservice  operational  test  and  evaluation 
(MOT&E)  were  evaluated  using  the  SE.  The  results  of  this  evaluation  is  included  as 
Appendix  G:  Section  G2. 

d)  Evaluate  Test  Results:  The  primary  deviation  of  the  SE  from  the  real  world,  realistic  vehicle 
movement,  has  previously  been  discussed  and  solutions  have  been  proposed.  Another 
comment  about  the  SE  deviation  from  the  real  world  was  made  by  an  Air  Force  operator  after 
performing  a  six-hour  mission  in  a  mock-up  of  the  E-8C.  To  paraphrase  him,  I  knew  it  wasn’t 
real  because  it  didn’t  stink  like  jet  fuel  and  the  plane  didn’t  roll  when  we  turned.  Other  than 
that,  it  was  Joint  STARS. 

Aside  from  the  fact  that  the  mock-up  of  the  E-8C  does  not  stink  like  jet  fuel  and  does  not  roll, 
reviews  of  log  sheets,  operator  comments  and  test  reports  reveal  that  the  ETE  Test  SE  is  a 
realistic  representation  of  the  real  world. 

As  an  aside,  it  is  almost  impossible  to  compare  the  SE  as  a  whole  to  the  real  world  because  the 
real  world  has  never  existed.  The  ETE  Test  concept  calls  for  a  major  land  battle  involving 
nearly  10,000  potential  targets,  at  which  ATACMS  missiles  are  targeted  and  fired.  None  of 
the  fielded  and  simulated  systems  within  the  ETE  Test  have  ever  been  used  under  these 
conditions.  The  closest  they  have  come  to  this  environment  was  post  Desert  Storm 
Southwest  Asia  (SWA)  deployments  and  Bosnia.  Thus  the  real  world  becomes  what 
operators  and  subject  matter  experts  think  will  happen,  or  hope  will  happen. 

e)  Evaluate  Operator  Performance:  Generally,  the  operators  performed  as  well  in  the  SE  as 
they  would  perform  in  the  real  world.  As  previously  mentioned,  this  issue  is  complicated  by 
the  fact  that  operators  throughout  the  SE  were  able  to  perform  tasks  that  they  had  never  been 
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able  to  perform  in  the  real  world.  Also,  as  previously  mentioned,  the  operator's  level  of 
training  and  the  ability  to  train  complicated  this  issue,  since  the  ETE  Test  SE  turned  out  to  be 
the  best  trainer  available. 

Evaluations  of  operator  performance  during  the  risk  reduction  test  are  contained  in  Appendix 
F:  Operator  Observer  Logs  and  Appendix  G:  Section  Gl:  Operational  Measures  Report.  In 
summary,  within  the  constraints  of  the  environment  portrayed  by  the  SE,  anything  the 
operators  could  do  in  the  real  world  they  could  do  in  the  SE. 

6.  Conclusion 

This  verification  and  validation  report  covers  the  final  four  steps  of  the  DIS  Nine  Step  W&A 
Process  as  referred  to  in  the  Recommended  Practice  for  Distributed  Interactive  Simulation  — 
Verification,  Validation,  and  Accreditation ,  4  November  1996  and  completes  the  verification  and 
validation  of  the  ETE  Test  synthetic  environment  for  the  ETE  Phase  2  operational  Test.  In 
addition,  it  completes  the  verification  and  validation  of  the  individual  simulation  nodes  of  the  ETE 
Test  SE  for  the  ETE  Phase  2  operational  test. 

The  results  of  this  phase  of  the  V&V,  along  with  the  results  of  the  Phase  1  V&V,  will  be  applied 
to  the  requirements  and  acceptability  criteria  for  the  ETE  Test  and  a  recommendation  will  be 
made  as  to  whether  the  ETE  Test  SE  should  be  accredited  for  use  during  the  ETE  Phase  2 
operational  test. 

A  review  of  the  results,  however,  would  indicate  that  the  ETE  Test  SE  is  a  very  robust  and 
realistic  environment  except  for  the  movement  problem  discussed  earlier.  Since  two  possible  and 
one  absolute  solution  to  this  problem  exist,  it  would  appear  that  the  ETE  Test  SE  is  ready  to  be 
used  for  the  ETE  Phase  2  operational  test. 
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Acronyms  and  Definitions 


ACE 

analysis  and  control  element 

ADS 

advanced  distributed  simulation 

AFATDS 

Advanced  Field  Artillery  Tactical  Data  System 

ATACMS 

Army  Tactical  Missile  System 

ATWS 

Advanced  Technology  Work  Station 

COTS 

commercial  off-the-shelf 

DIS 

distributed  interactive  simulation 

ES 

entity  state 

ES  PDU 

entity  state  protocol  data  unit 

ETE 

End-to-End  Test 

Force 

a  menu  option  that  opens  the  Scenario  Forces  Editor  which  is  used  to 
modify  system  numbers,  aggregation  and  task  force  assignments  for  a 

Janus  scenario 

HQ 

headquarters 

IEEE 

Institute  of  Electrical  and  Electronics  Engineers 

JADS 

Joint  Advanced  Distributed  Simulation,  Albuquerque,  New  Mexico 

Janus 

interactive,  computer-based  simulation  of  combat  operations 

Joint  STARS 

Joint  Surveillance  Target  Attack  Radar  System 

km 

kilometer 

LGSM 

light  ground  support  module 

M&S 

modeling  and  simulation 

MOT&E 

multiservice  operational  test  and  evaluation 

MTI 

moving  target  indicator 

OT 

operational  test 

PCNIU 

personal  computer  network  interface  unit 

PDU 

protocol  data  unit 

SAR 

synthetic  aperture  radar 

SATCOM 

satellite  communications 

SE 

synthetic  environment 

SME 

subject  matter  experts 

Spectrum™ 

an  instrumentation  suite  used  to  measure  bandwidth  utilization 

STRICOM 

U.S.  Army  Simulation,  Training  and  Instrumentation  Command 

SWA 

Southwest  Asia 

TAC 

target  analysis  cell 

TACFIRE 

tactical  fire 

TAFSM 

Tactical  Army  Fire  Support  Model 

TCAC 

Test  Control  and  Analysis  Center 

TRAC 

U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Analysis 
Center 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 

V&V 

verification  and  validation 

VSTARS 

Virtual  Surveillance  Target  Attack  Radar  System 

21 


W&A 

WSMR 


verification,  validation  and  accreditation 
White  Sands  Missile  Range 
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Appendix  A:  Janus  Suite  Price  List 


Suggested  Equipment  For  Janus 
As  VSTARS  Stimulator 


Model  Base  Product  A43 1 8A 

C200  200  MHz  PA-RISC  8200  CPU  $13,000 

Configured  with: 

4GB  FWD  SCSI-2  low  profile  disk  drive  (1.6”  high)  1 ,700 

4GB  DDS-2  DAT  drive  1 ,400 

HP-UX  10.x  Instant  Ignition  200 

Keyboard  100 

128  MB  ECC  memory  module  900 

HP  VISUALIZE-EG  card  (requires  1  GSC  slot)  1,000 

19”/ 18.3  viewable  color  monitor  1,300 

Cable  for  DDS-2  drive  100 

Terminator  for  cable  50 

total  estimate  19,800 


Several  options  exist:  the  256  MB  ECC  memory  module  is  about  $3K,  both  larger  and  smaller 
monitors  are  offered,  an  additional  disk  drive  can  be  added,  and  you  should  check  with  your  local 
systems  support  for  what  you  need  for  networking.  We  use  twisted  pair  cables,  some  use  thin 
LAN,  etc.  You  may  need  extra  cards.  You  might  want  the  OS  media  as  well  as  the  preloaded  we 
have  shown  here;  if  so,  you  will  need  a  CD-ROM  drive.  I  used  the  HP  web  site  for  some  of  the 
above  info:  see  http://www.hp.  com/cgi-bin/hpwebcat/config3.pl/hp9kw. 

Different  disk  drives  might  be  chosen  to  suit  your  connections.  Consult  with  HP  experts  to  make 
sure  of  system  compatibility  and  connecting  compatibility. 

The  C240  with  a  236  MHz  processor  is  out  and  costs  about  $5K  more. 


Al- 
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Appendix  B:  Analysis  of  Janus  PDUs 


Sample  of  071398  WSMR  (Janus)  PDU  Analysis 


Log  Time 

Game 

Entity 

,  i  Velocity  X 

Velocity  Y 

Velocity  Z 

1 

UTM 

UTM 

Site  Host  *  x 

(msec) 

Time 

(msec) 

Latitude 

Number 

Longitude  .  . 

B  (m/sec) 

(msec) 

I  (msec) 

Northing 

Easting 

[E£IE£i 

ImEU 


18000472 

65406113 

18000472 

65406113 

18000472 

65406113 

18000472 

65406114 

18000806 

65406114 

18000806 

65406114 

18000806 

65406114 

18000806 

65406119 

18001138 

65406119 

18001138 

65413137  18007804 


65413137  18007804 
65413137  18008138 


65413137  18008138 


65413137  18008138 
65413137  18008138 


I  »*■■:!■ 


65484119 


65484119 


65484119 


65484124 


65484124 


65484124 


65484124 


65549104 

65549105 


65549105 


65549105 


65550094 


65550094 


65550095 


65550095 


65550095 


18078786 


18078786 


18078786 


18079120 


18079120 


18079120 


18079120 


18144104 


18144104 


18144104 


18144104 


18144438 


18144438 


18144438 


18144438 


18144438 


0 

3379864 

632062 

0 

631939 

0 

632022 

0 

3379910 

632236 

0 

3379933 

632322 

0 

3379862 

632287 

0 

3379930 

631896 

0 

3379969 

631882 

0 

3379842 

631831 

0 

3379770 

631881 

-4.74379 

336540 

630427 

-4.74505 

336546 

630426 

-4.74503 

336551 

630425 

-4.96061 

336561 

630423 

-4.74496 

336566 

630422 

-4.74414 

336459 

630427 

31.5318 


31.5327 


31.5332 


31.534 


46.198661  2.88727  |  1.207753 


46.19878  2.887297 


46.19901  2.88735 


2.887377 


| -4.67828  | 

348916 

613773 

348926 

613794 

|  -4.67763  1 

348931 

613805 

1.20786 


2.794934  1.115418 


1.207833  1-4.67758 


-4.67756 


-4.4644 


3836|  31.53581  46.1 9948 1  2.887456  \  1.20794  j-4. 67749 


31.1173 
31.1173  ' 


31.1173 


31.1173 


31.1173 


31.1173 


31.1173 


31.1173 


31.1173 


46.42403 


46.42403 


46.42403 


46.42403 


46.42403 


46.42403 


46.42403 


46.42403 


46.42403 


-3.83013 

-3.83013 


-3.83013 


-3.83013 


-3.83013 


-3.83013 


-3.83013 


-3.83013 


-3.83013 


-2.76952 

-2.76952 


-2.76952 


-2.76952 


-2.76952 


-2.76952 


-2.76952 


-2.76952 


-2.76952 


7.69744 

7.69744 


7.69744 


7.69744 


7.69744 


7.69744 


7.69744 


7.69744 


7.69744 


348941 


348946 


348955 


348960 


344341 


344342 


344343 


613826 


613836 


613858 


613868 


635787 


635787 


635787 


344345 


344346 


344347 


344348 


344349 


635787 


635787 


635787 


635787 


635787 


BIE 

m 

65550095 

18144770 

3 

1.1 

173 

4 

6.42403 

-3.83013 

-2.76952 

7.69744 

344343 

635787 

mm 

m 

65550095 

18144770 

|  3 

1.4 

357 

4 

7.25588 

0 

0 

0 

3480091 

714400 
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Appendix  C:  Section  Cl:  Janus  Monitor  Results 


Cl- 


TAFSM  Entity  Locations  -  Vignette  3  (Scenario  903) 


STATE  PDU  outside  box  for  Unit  9905 
XLL,  UTM_X,  XUR  =  545.0  717.4  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3299.0  3545.0 
STATE  PDU  outside  box  for  Unit  9906 
XLL,  UTM_X,  XUR  =  545.0  710.5  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3293.5  3545.0 
STATE  PDU  outside  box  for  Unit  9907 
XLL,  UTM_X,  XUR=  545.0  712.2  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3297.0  3545.0 
STATE  PDU  outside  box  for  Unit  9908 
XLL,  UTM_X,  XUR  =  545.0  714.5  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3296.5  3545.0 
STATE  PDU  outside  box  for  Unit  9909 
XLL,  UTM_X,  XUR  =  545.0  707.8  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3294.8  3545.0 
STATE  PDU  outside  box  for  Unit  9910 
XLL,  UTM_X,  XUR  =  545.0  708.1  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3295.0  3545.0 
STATE  PDU  outside  box  for  Unit  991 1 
XLL,  UTM_X,  XUR  =  545.0  708.1  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3294.6  3545.0 
STATE  PDU  outside  box  for  Unit  9912 
XLL,  UTM_X,  XUR  =  545.0  709.6  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3296.1  3545.0 
STATE  PDU  outside  box  for  Unit  9913 
XLL,  UTM_X,  XUR  =  545.0  709.9  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3296.3  3545.0 
STATE  PDU  outside  box  for  Unit  9914 
XLL,  UTM_X,  XUR  =  545.0  709.9  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3295.9  3545.0 
STATE  PDU  outside  box  for  Unit  9915 
XLL,  UTM_X,  XUR  =  545.0  709.5  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3294.2  3545.0 
STATE  PDU  outside  box  for  Unit  9916 
XLL,  UTM_X,  XUR  =  545.0  709.8  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3294.4  3545.0 
STATE  PDU  outside  box  for  Unit  9917 
XLL,  UTM_X,  XUR  =  545.0  709.8  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3294.0  3545.0 
STATE  PDU  outside  box  for  Unit  9918 
XLL,  UTM_X,  XUR  =  545 .0  7 1 1 . 1  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3296.7  3545.0 
STATE  PDU  outside  box  for  Unit  9919 
XLL,  UTM_X,  XUR  =  545.0  71 1.4  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3296.9  3545.0 
STATE  PDU  outside  box  for  Unit  9920 
XLL,  UTM_X,  XUR  =  545.0  711.4  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3296.5  3545.0 
STATE  PDU  outside  box  for  Unit  9921 
XLL,  UTM_X,  XUR  =  545.0  710.9  770.0 
YLL,  UTM_Y,  YUR  =  3320.0  3298.4  3545.0 
STATE  PDU  outside  box  for  Unit  9922 
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Appendix  C:  Section  C2:  Location  of  Fort  Sill  Entities  as  Supplied  by  TAFSM 
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stdin 

Entity 

MGRS  ; 

LATLONG 

• 

UTM 

i 

EASTING 

NORTHING 

ZONE 

FDC:?SE 

38RQU190019? 

29 

49 

41.39 

N,  47 

15 

59.27 

E; 

719100 

3301900 

38 

FDC: 1/14 

38RQU191026; 

29 

50 

4.05 

W.  47 

16 

3.51 

E? 

719100 

3302600 

38 

FDC: A/1/14 

38RQU192034; 

29 

50 

29.95 

N,  47 

16 

7.82 

E; 

719200 

3303400 

38 

FU: 1/1/A/l/ 14 

38RQUI91031; 

29 

50 

20.28 

N,  47 

16 

3.88 

E; 

719100 

3303100 

38 

FU: 2/1/A/1/14 

38RQU189034; 

29 

50 

30.14 

N,  47 

15 

56.65 

Ej 

718900 

3303400 

38 

FU: 3/1/A/1/14 

38RQU193G04;> 

w  4 

rifL 

CL 

£ 

2J— 5r 

1*300 

7?,sV 

7 

J  J  i/ a  vv 

Vr 

FV : 1 /2/A/ 1/14 

38RQU202035; 

29 

50 

32.56 

N,  47 

16 

45.13 

B; 

720200 

3303500 

38 

FU:2/2/A/l/14 

38RQU200038; 

29 

50 

42.43 

N,  47 

16 

37.91 

2; 

720000 

3303800 

38 

FU : 3 /2/A/ 1/1 4 

38RQU204038; 

29 

50 

42 . 17 

N,  47 

16 

52.80 

E; 

720400 

3303800 

38 

FU :  1/3 /A/ 1/14 

38RQU181021; 

29 

49 

48.45 

N,  47 

15 

25.91 

E; 

718100 

3302100 

38 

FU : 2/3/A/1/14 

38RQU179024; 

29 

49 

58.32 

N,  47 

15 

18.68 

E; 

717900 

3302400 

38 

FU: 3/3/A/1/14 

38ROU183024; 

29 

49 

58.06 

N,  47 

15 

33.58 

B; 

718300 

3302400 

38 

FDC :B/1/14 

38RQT180990; 

29 

48 

7.88 

N,  47 

15 

19.93 

E; 

718000 

3299000 

38 

FU : l/l/B/1/14 

38RQT168988; 

29 

48 

2.14 

N,  47 

14 

35.11 

E; 

716800 

3298800 

38 

FU: 2/1/B/1/14 

38RQX166991; 

29 

48 

12.01 

N,  47 

14 

27.89 

B; 

716600 

3299100 

38 

FU: 3/l/B/l/l4 

38RQT170991; 

29 

48 

11.76 

N,  47 

14 

42.78 

E? 

717000 

3299100 

38 

FU: 1/2/B/1/14 

38KQU178012? 

29 

49 

19.43 

N,  47 

15 

14.08 

E; 

717800 

3301200 

38 

FU: 2/2/B/1/14 

38RQU17601 $7 

29 

49 

23.29 

N,  47 

15 

6.86 

E? 

717600 

3301500 

38 

FU:3/2/B/l/14 

38RQU1B0015; 

29 

49 

29.04 

N,  47 

15 

21.75 

E? 

718000 

3301500 

38 

FU: 1/3/B/1/14 

38RQT16898Q; 

29 

47 

36.17 

N,  47 

14 

34.53 

E; 

716800 

3298000 

38 

FU:2/3/B/l/14 

38RQT166983,- 

29 

47 

46.04 

N,  47 

14 

27.31 

B? 

716600 

3298300 

38 

FU: 3/3/3/1/14 

38RQT170983; 

29 

47 

45.79 

N,  47 

14 

42.20 

E; 

717000 

3298300 

38 

FDC: C/1/14 

38RQU224015; 

29 

49 

26.21 

N,  47 

18 

5.57 

E; 

722400 

3301500 

38 

FU:1/1/C/1/14 

38RQU316028; 

29 

50 

2.32 

N,  47 

23 

49.08 

E; 

731600 

3302800 

38 

FU : 2/1/C/1/14 

38RQU3 14031? 

29 

50 

12.19 

N,  47 

23 

41.86 

E; 

731400 

3303100 

38 

FU:3/l/ C/l/14 

38RQU318031; 

29 

50 

11.92 

N,  47 

23 

56.76 

E; 

731800 

330310Q 

38 

FU : 1/2/C/1/14 

38RQU329028; 

29 

50 

1.44 

N,  47 

24 

37.48 

E; 

732900 

3302800 

38 

FU:2/2/C/l/l4 

3&RQU327031; 

29 

50 

11.31 

N,  47 

24 

30.26 

E; 

732700 

3303100 

38 

FU: 3/2 /C/l/14 

38ROU331031? 

29 

50 

11.04 

N,  47 

24 

45.16 

E; 

733100 

3303100 

38 

FU: 1/3/C/1/14 

38RQU306013; 

29 

49 

14.30 

N,  47 

23 

10.69 

E; 

730600 

3301300 

38 

FU: 2/3/ C/l/14 

38RQU304016; 

29 

49 

24.17 

N,  47 

23 

3.47 

E; 

730400 

3301600 

38 

FU: 3/3/c/l/l4 

38RQU3Q8Q16  7 

29 

49 

23.90 

N,  47 

23 

18.36 

E; 

730800 

3301600 

38 

1 

/ 

( 

t  Or  £>*\  “To 
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1 

e>^ 
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| Entity  MGRS  ;  LAT-LONG 


I D : 4  054  427 140 


?  UTM 


FDOrF^E 

FDC : 1/14 

FDC: A/l/l4 

FU : l/l/A/1/14 
FU : 2/1/ A/1/ 14 
FU: 3/1/A/1/14 

FU : 1/2 /A/ 1/1 4 
FU: 2/2/A/1/14 
FU: 3/2/A/1/14 

FU: 1/3 /A/ 1/1 4 
FU:2/3/A/l/l4 
FU:3/3/A/l/l4 

FDC ; B/l/14 

FU: l/l/B/1/14 
FU:2/1/B/1/14 
FU:3/l/B/l/l4 

FU; 1/2/b/I/ 14 
FU: 2/2/B/1/14 
FU : 3/2/B/ 1/ 14 

FU : l/i/B/I/14 
FU:2/3/B/l/l4 
FU : 3/3/B/l/ 14 


FDC: C/1/14 

FU: l/l/C/l/14 
FU:2/1/C/X/14 
FU: 3/1/C/1/X4 

FU:l/2/c/l/l4 
FU: 2 /2/C/ 1/14 
FU:3/2/C/l/14 

FU : 1/3 /C/1/14 
FU : 2/3 /C/1 /14 
FU: 3/3/ C/1/ 14 


Bast.  ; 

North. 

Zone 

38RQT29  Q754 ; 

29 

35 

14.60 

N,  47 

21 

51.40 

E;  729000 

3275400 

38 

33RQT299756 ; 

29 

35 

20.50 

N,  47 

22 

24.98 

E;  729900 

3275600 

38 

38RQT27077S; 

29 

36 

24.09 

N,  47 

20 

38.69 

E;  727000 

3277500 

38 

38RQT258785; 

29 

36 

57.34 

N,  47 

19 

$4.86 

E;  725800 

3278500 

38 

38RQT260788; 

29 

37 

6.95 

N,  47 

20 

2.51 

E;  726000 

3278800 

38 

38RQT262785; 

29 

36 

57.08 

N,  47 

20 

9.72 

E;  726200 

3278500 

38 

38RQT279794; 

29 

37 

25. 18 

N,  47 

21 

13.  S6 

E;  727900 

3279400 

38 

38RQT281797; 

29 

37 

34.79 

N,  47 

21 

21.22 

E;  728100 

3279700 

38 

38RQT283794,- 

29 

37 

24.91 

N,  47 

21 

28.42 

E;  728300 

3279400 

38 

38RQT277778; 

29 

36 

33.37 

N,  47 

21 

4.92 

E;  727700 

3277800 

38 

38RQT279781; 

29 

36 

42.98 

N,  47 

21 

12.58 

E;  727900 

3278100 

33 

38RQT281778; 

29 

36 

33.11 

N,  47 

21 

19.7$ 

E;  728100 

3277800 

38 

38RQT315753; 

29 

35 

9.69 

N,  47 

23 

24.17 

E;  73150Q 

3275300 

3$ 

38RQT298767; 

29 

35 

56.27 

N,  47 

22 

22.10 

E;  729800 

3276700 

3$ 

38RQT300770? 

29 

36 

5.88 

N,  *7 

22*29.76 

B;  730000 

3277000 

38 

38RQT302767; 

29 

35 

56.01 

N,  47 

22' 

36.96 

E;  730200 

3276700 

38 

38RQT321776; 

29 

36 

23.95 

N,  47 

23 

48.22 

E;  732100 

3277600 

38 

38RQT323779; 

29 

36 

33.56 

N,  47 

23 

55.88 

E;  732300 

3277900 

38 

38RQT325776; 

29 

36 

23.68 

N,  47 

24 

3.08 

E;  732500 

3277600 

38 

38RQT318758; 

29 

35 

25.72 

Nf,  47 

23 

35.70 

E;  731800 

3275800 

38 

38RQT320761; 

29 

35 

35.33 

N,  47 

23 

43.36 

E;  732000 

3276100 

38 

38RQT322758; 

29 

35 

25.45 

N,  47 

23 

50.56 

E;  732200 

3275800 

38 

38RQT355750; 

29 

34 

57.25 

N,  47 

25 

52.50 

E;  735500 

3275000 

38 

38RQT341761; 

29 

35 

33.91 

N,  47 

25 

1.36 

B;  734100 

3276100 

38 

38RQT343764? 

29 

35 

43.51 

N,  47 

25 

9.02 

E;  734300 

3276400 

38 

38RQT345761; 

29 

35 

33.64 

N,  47 

25 

16.21 

E;  734500 

3276100 

38 

38RQT365773; 

29 

36 

11.23 

N,  47 

26 

31.43 

E;  736500 

3277300 

38 

38RQT367776; 

29 

36 

20.83 

N,  47 

26 

39.10 

E;  736700 

3277600 

38 

38RQT369773; 

29 

36 

10.96 

N,  47 

26 

46.29 

E;  736900 

3277300 

38 

38RQT361751; 

29 

35 

0.09 

N,  47 

26 

14.86 

E?  736100 

3275100 

38 

38RQT363754; 

29 

35 

9.69 

N,  47 

26 

22.52 

E?  736300 

3275400 

38 

38RQT365751; 

29 

34 

59.82 

N,  47 

26 

29.71 

E?  736500 

3275100 

38 

^  V  ^ 
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Entity  MGRS  ;  LAT-LONG  ;  UTM  ’ 


East.  Worth.  Zone 

FDC : F$E 

38RQT186990; 

29 

48 

7.49 

N,  47 

15 

42.26 

B;  718600 

3299000 

38 

FDCsX/14 

38RQT17499Q; 

29 

48 

8.26 

N,  47 

14 

57.59 

E;  717400 

3299000 

38 

FDC:A/1/14 

38RQT105935; 

29 

45 

14,00 

N,  47 

10 

36.88 

E;  710500 

3293500 

38 

FU : 1/1/A/ 1/14 

38ROT078948; 

29 

45 

57.85 

N,  47 

8 

57.32 

E;  707800 

3294800 

38 

FU :  2/1/ A/ 1/1 4 

38RQT081950; 

29 

46 

4.16 

N,  47' 

9 

8.62 

E;  708100 

3295000 

38 

FTJ : 3/1/A/1/14 

38RQT081946; 

29 

45 

51.17 

N,  47 

9 

8.34 

E;  708100 

3294600 

38 

FU : 1/2 /A/ 1/1 4 

38RQT095961; 

29 

46 

38.96 

N,  47 

10 

5.21 

E;  709600 

3296100 

38 

FU: 2/2/A/1/14 

38RQT099963; 

29 

46 

45.27 

N,  47 

10 

(16 . 51 

S;  709900 

3296300 

38 

FU : 3/2 /A/ 1/1 4 

38RQT099959; 

29 

46 

32.28 

N,  47 

10 '<16. 23 

E;  709900 

3295900 

38 

FU: 1/3 /A/ 1/14 

38RQT095942; 

29 

45 

37.33 

H,  47 

10 

0.16 

E?  709500 

3294200 

38 

FU:2/3/A/l/l4 

38ROT098944? 

29 

45 

43.65 

N,  47 

10 

11.46 

E;  709800 

3294400 

38 

FU; 3/3 /A/ 1/14 

i 

38RQT098940; 

29 

45 

30.66 

N,  47 

10 

11.18 

E;  709800 

3294000 

38 

rDCtB/1/14 

38RQT122970; 

29 

47 

6.58 

W,  47 

11 

42.61 

E;  712200 

3297000 

38 

FU : l/l/B/l/14 

38ROT111967; 

29 

46 

57.52 

N,  47 

11 

1.46 

E;  711100 

3296700 

38 

FU: 2/1 /S/1/1 4 

38RQT114969; 

29 

47 

3.83 

N,  47 

11 

12.7$ 

E;  711400 

3296900 

38 

FU:3/1/B/1/14 

38RQT114965; 

29 

46 

50.84 

N,  47 

11 

12.48 

E;  711400 

3296500 

38 

FU: 1/2 /B/ 1/14 

33RQT109984; 

29 

47 

52.84 

N,  47 

10 

55.21 

S ;  710900 

3298400 

38 

FU: 2/2 /B/ 1/14 

38RQT112986; 

29 

47 

59.14 

N,  47 

11 

6.52 

E;  711200 

3298600 

38 

FU: 3/2/B/1/14 

38RQT112982; 

29 

47 

46.16 

N,  47 

11 

6.21 

E;  71120G 

3298200 

38 

FU: 1/3/B/1/14 

33RQT126989; 

29 

48 

8.02 

N,  47 

11 

58.85 

E;  712600 

3298900 

38 

FU: 2/3/B/1/14 

38RQT129991; 

29 

48 

14.33 

N,  47 

12 

10.16 

E;  712900 

3299100 

38 

FU: 3/3/B/1/14 

38RQT129987; 

29 

48 

1.34 

N,  47 

12 

9.87 

E?  712900 

3298700 

38 

FDC: C/1/14 

38RQT14596S; 

29 

46 

48.92 

N,  47 

13 

7.85 

E;  714500 

3296500 

38 

FU: l/l/C/1/14 

38RQT134972; 

29 

47 

12.33 

N,  47 

12 

27.41 

E;  713400 

3297200 

38 

FU: 2/1/C/ 1/14 

38RQT137974 ; 

29 

47 

18.64 

N,  47 

12 

38.72 

E;  713700 

3297400 

38 

FU : 3/1/C/ 1/14 

38RQT137970; 

29 

47 

5.65 

N,  47 

12 

38.44 

E;  713700 

3297000 

38 

FU : 1/2/C/1/14 

38RQT152990; 

29 

48 

9.64 

N,  47 

13 

35.70 

E;  715200 

3299000 

38 

FU:2/2/C/l/14 

38RQT1S5992; 

29 

48 

15.95 

N,  47 

13 

"47.01 

E;  7iS500 

3299200 

38 

FU: 3 /2/C/ 1/14 

38RQT155988; 

29 

48 

2.96 

N,  47 

13 

46.72 

E;  715500 

3298800 

38 

FU : 1/3/C/1/14 

38RQT153969; 

29 

47 

1.41 

N,  47 

13 

37.91 

E;  715300 

3296900 

38 

FU:2/3/C/l/14 

38RQT156971; 

29 

47 

7.71 

N,  47 

13 

49.22 

E;  715600 

3297100 

38 

FU:3/3/C/l/14 

38RQT156967; 

29 

46 

54.72 

N,  47 

13 

48.93 

Bj  715600 

3296700 

38 

Wednesday  July  08,  98 


stdin 


Appendix  C: 

Section  C3:  TRAC-WSMR  Log  Sheets 


ETE  EVENT  LOG 


Time 

(Zulu) 


Location :  'jJ£M  _ 

Test  Event  /?/Z _ 

Test  Date  T  < 

Recorder  Name:  A 

Comments 


IcrD 

^Vl 

-  /cfb  r7*  /TV/ 


^  (7  "  ,  0  2.  O  j-g-c _ _ 


Sill  j+sit  r*ee„n*j 

a jan^sLrUJc-s _  /njJUJ _ 


4^/^g  ^,0 


/  • 

2>u  S~7  ;j~7  /  t  "i  :  s  c>  '.  rfc 


jk>D^ 


i?3 


107- 


1M 


fflo 1 


'23^ 


1  </£  «" 


i7:  0 


I  n :  <4iclr 


9>'.do  ; 


k:*V  / 
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C  5 


ETE  EVENT  LOG 


Location  _ 

Test  Event  _ 

Test  Date  7  4wt- 
Recorder  Name:  f+dr  Aru.ic^' 

Time 


(Zulu)  Comments 


C-3 


ETE  EVENT  LOG 


Time 

(Zulu) 


Location:  £ 

Test  Event  £jt_ _ 

Test  Date  ft  ,.}ul  9 g 

Recorder  Name:  . 


Comments 


L<r\  A-/?  /  (y)Ju 

nfi 

J  i  I  '  /  /  ,  , 

tCSt  /  fjb^c  [ t*sSr%*s  rn*Ay  7^  £%-$iUs  ^ 

nrr 

)  j  . 

Cb-t<M'  ~~  r  00  O  ll  3  &*-<-■ 

/  7 

\J'p~db  1  !^S>°n  G^hkjj  ^/r/djS'f  j  jZS  -  z_ 

nrs 

C'Luscjcs  (ovJhyvHCG  CcJ^^ 

(3o3 

1  l  i  fa  rnJ-*<  (rrAit**^  eJ-  s'M(i'JeSt) 

rluok-*  M<~,rniojr') 

ins' 

SU-\ ri-.fi  /  <**/<»-  / 

\m 

r  )dMUi  I’.IJ'  j  /  J.  yg  ll-lA 

i4T» 

J&m*./  TJ.'l&l j  H;y°  tl^r- 

<  jiU^(  iJcrtjJtir?  llotontJastilb^MlSS/fxi  /tv  7:SY-*H 

/S'/I 

- r*ri  l - ^ - 

r-l  (^u//  5  :  C?  '.  r*7  /  /  S"*;  /  c/ '.  !  J 

/ryr 

(jCUvuu/  J  /  /f  s  YS~:  /7?r 

/td 

-5/W  (/:o!.’St7  / /Lt  rt-is  zr 

MC 

0\  j 

J<W4  4;?,^  f*?  j  /!e  Vf;  /J?- 

n «r~ 

oLm  ^7  /-7.-/r:  ,7j- 

Y7 

Jcsv^j  ± ;  ??—  j  n  ■.  yi : 

ETE  EVENT  LOG 

Location:  cJSM  £ _ 

Test  Event  PP _ 

Test  Date  g  Juu  98 
Recorder  Name:  Cnof  j-W* 


Comments 


^  6/  ?y,v 


SjuhLuj*  ftp  l  fib  svjsicvH  beQuir ^  eJx  t 
~^pshrF~i^vutO  i  'fS-~  j  1  7__ 


J <&h**  P/-T  ^ '  z.  H‘ie%  & :  ?y- 


Page _ 


ETE  EVENT  LOG 


Time 

(Zulu) 


Location :  /AS  ^ 

Test  Event _ jj 

Test  Date  7  Jut 
Recorder  Name:  Cc^t 


Comments 


nro 

n  rr 

;  - ■ j  t 

\tfci70l<ris  WMtJv'  V*  fhiil  l  u+s 

i  t  rs 

1  J  / 

—.oooO’lZ  sec. 

i  ?ov 

^rrJkZ-sJ^.cIoz  /97f7*J,W^r°  ! 

/  Jcrz> 

- 7* - ■  f - 1 - ‘ - / - f 

{/cecAtf-fas  nihstric-  c-l^ks .  Ccdre*^  hd&JLtJ. 

\3°4 

A Cl^ob  - 

m3 

dtv-l  ?  7c1i 

lV^ 

JtdJfm  D3  yyT3  l  137 Cm O-^- 

/•337 

(338 

/J?7 

&'l*?up  ' 

[lidded  f  ,  CtiA^uv-b  °l&l  l  /3S-  9*7 

13*3 6 

1 

/?57 

D  is  r^.  r^/  n.-nr^Tr 

J&ivus  t:3&  /  i'-(:zcj  :  ‘To  z- 

/4<r 

j  k-m  j  Z  ■  c  c  /  /c/  :  ^  /  jg. — 
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C-6 


ETE  EVENT  LOG 


Time 

(Zulu) 


Location:_ 
Test  Event 
Test  Date 


i  USm/cL. 


Recorder  Name: 


Comments 


AW*  r£ceiu\><l  TAHM  fahO-gc  iwcjUj)  . 

'  *tfXiv(0>J)hy  IrtJrunr- 

'T/VfrM  aj  e if  ctefai^ Jh<~ 

/^/*y  ~H>  £'7cft  A  ^1*^011  /  _  +tc*W 

<31 

J<W«7  z:y<J  si  /  fr:-3t-:SZT 

rjzw/  7-  •■  -^7 .'  r  7  /  /.TV  />  ,*  r?  ^ 

MB 1 

1  .  VS:  n  /  /6  :  3  7;  Ti 

r)^v^/  'V  '  C-7  '  5^*7  /  /C>  :.  ^  ~is- 

11 IZ- 

- — '-■ - 7 — — 1 - 

\.«o^s  tf;ieI.T< j  /  /7  ;  2. /  ;  rZ-^2_ 

Wol 

t  1  * 

<<^i  r:  K  j  (S:t~L'.  S"Z  7r~ 

1 

/95‘^- 

ciw-i  7 :  /  0  r  y  0 

"Zoo  3 

(r4‘ yts  33  SSVS  &vJ\li?S  J>M*S 

"j2e  ^  J1  S'5  JUn^l e  jd^  j  / J  kllr  —  /?// -fra «UnJ(  *6 

-7.0  i  0 

^  ^  [traruJr  <Ju 

7 )  -  — 7 
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C-7 


ETE  EVENT  LOG 


M 


Time 

(Zulu) 


Location 
Test  Event 
Test  Date  jp  ,luc 
Recorder  Name:  VguM/ 


Comments 


/  300 

nr  4  Ls-^  ov\  /^Jy 

j  30  3 

61% jcp  t  u^~ 

T  J 

cUL  ^  b(^  “ ,  cocoin  ( 

f\J  fw  k  rv+wtrtfrz, ib.Ks;  cJ-rCAC 

(  lo  L> 

\hpji  J  ,Jto+«'o  963  /  ‘fo'feJL  l3lc/Uo 

1 

-"W  %  '  ttu‘  J/'*#-TV  !</f: iS*e  Lift-  /  Ji^rr  ^r)  ^  ><*•.  /x. 

(CnrCc^  L  L  70  •(*  -  sc  y  e/f  ift;/o  LtC  Jc  rr.'?/  i_ 

i  y$ 

■  .  . - - - - / 

J/?V  /■  ^ tjfOy'  01(0  yS  _  ?£/ /  fe?  0  /_  k  £ 

1730 

Civ'A  ^>K( /  o;oo  /  /? :  T  o  ;c  6 

pro 

i  1  1 

v  Atw;  ^'J4>  /  /T : 

tiio 

c !oumS  ttrTl  /  rp.io-.c^*- 

i^i 

c  louv^w  / :  7  /  1  N  ■■  S'/  •  o  1  ?r 

/f?o 

/ 

^JduM  s  Z  ' era  /  / S  *  t-°-  i  0 

ipo 

r  r - - - 

c  kW  2‘.  /*  /  Jf:  ?o:  tv  ?r 

fM 

{nrf  (tCtiflP  0 orffy  y+w&L  )  'j2ec,( ^  J)  J^liridk 

(G  /? 

pH  IclcJrC-  C'ic.ci:'1^ 

ii’zo 

?:H:r  "7  /  /L  ;Z0  :  /OTr 

Page 


of 


7  * 

■  n. 


JVUV) 

h?s 


ETE  EVENT  LOG 


Time 

(Zulu) 


/W'o 


llZo 


nf° 


■7  rrl 


IBzo 


lb?* 


o 


£ 


'  "7  7  ( 


/Wo 


°140 


(i<r 


Zoo  I 


2  O'lO 


2021 


Location:  (ASmC _ 

Test  Event _ i2£- _ 

Test  Date  lo  Jm  9& 

Recorder  Name:  Cyl 

Comments 

W  1  a  /  /  4tbm  '<r£crf>s>  /&<$  "  " 

thud1  /fobf-su.-huf  rt>  "5  /  '  &  z  *5  y  Lf.L,ltf:7<r 


r)a,y^J  JZUlll  I  r 


rv  x  ?/  S' i  *  /  i~f  :  2.0  ;  ^ 


Janu  S  t±J±i £±J_  Qj_£l :  /o  7- 


-r3-»  yo7i.’v8,vfc/feie 

fTi* ivjv/r,  v6.jta.  /j  jc7J  £•  ?&z« 

-LiikXltai _ ?^rc  iV.  ijjZlMS.  2±x±_£&^J±  1£_L± 


■jt  n  ,f,  i/bury 

.7<-.a.y»;  w  r  jc  ~ 

jif.g.a  .j^  Vt  t±  <V 


</.■  r?  _  r±Jjs_±o_itc%- 

-  I  fl  I  f  7 ‘  <&)  s<  -rftrJ  A/-Atrv7  -^C 


<r  -/  rl  LJr*  ^  sc  **>  fa  pj^oz 

[nKi^c  lwJv<C'^  c^p-7  Apr  £af  ifat»rru*J  f±£  UjUkxL  pi/ , 7£  /t.7  VV 


f:?r.-o±  J  /Sr  sy ;  / o  -gr 


n/r 


ck^,  6;o?  )  t^?2/r/o 

T^TTtzttt) 


Y7  (YC 

,  •/<*  /Y  ST 


J  ^.pPiscStZr 


IV 


I  ? 


prr^f //;/!. 


yo'tS  sr  vwj£7?£  ,'^u-^x^ 

7*  3}  2  -L  ULlCi  trt 


/r 


JoaW  j 


!>‘kp  7 


'^wu  X 


//ao:  to 


W  j/D^reptb, 

~ ~  ~  ~  _  Wfc  7 
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36:  frl*) 

(iC.;t<i:t8  1 


ETE  EVENT  LOG 


t/  vf  £— 


j2£- 


Time 

(Zulu) 


/  4 s^/c> 


/tro 


HZo 


nf° 


-?rn 


/8X0 


/5/B 


I  *u< 


!il( 


1^30 


ci  40 


\if 


Zoo  I 


2010 


zoil 


Location :_ 

Test  Event _ 

Test  Date _ [O  Jai  98 

Recorder  Name:  6y/  Am ~A±A_ 

Comments 

1  A  j  /  3  *  is  4n*x  '5~cWpj 

'frvu-SU*c!J  ib 


uf  (Aur\s~e  '  fa  3/ :  or  21,  it-L'.iT:J<r 


rv .r _ H  ’  Z&;  ^ 


Zr~ 


<2-,  ?*r^8. V*fT'&  ?6  JJ  'f  </c+7isr</ 

[fl?  S  j %73  C'qU7C>ze  d  **  ?  «*7  VI  <=  «?<C  - 


</  .■  r? _ n  ! ts  io  :  1 0 .£- 

TTfsTrS' 


Xx  ~77rZZ3  JcUtztiz? 

[n^ti^c  \Jnr«?><r'  c^p-7  ^2£  g^r  4t)  t*rn**J?  t± £  /*£*- 


£J2L^1  /  /^sT;/oe 

~J  "  f'lvb.t-V'tZ) 


C7/T 


6;pi  LilUiLLiR 


Ji^H 

prrriiii 


r°  y/  'vc  ^ 
7,  ■/fc  /v  r- 


I  J 


~^c77f¥r  'yUJU?Z  #,c£ 

7e  2  t,  c/Z  J6.  tr' 


/J~ 


^ '  3z^  f  !ct  -  f>  :  j  O  ^tr 

- - f - — - _ — .ittyv*s>  rCT  £!£&££.  ^ 


^Wuu  -T 


<  :oQ :  Jt> 


sh>  hyy*-  y/wrep/L, 

"Page  Z  of  l  ' 


g»a7t, 

7eC("fte‘~~r , 
Jetry^T' 


l-fS 


iV/fc3.i?3 

fV  /  78  (1-7  VV 


/V  ^  Wt) 

j  ^;iM78  ! 


psi* 

ftx.hr 


CIO 


ETE  EVENT  LOG 


Time 

(Zulu) 


Location:  UJSjijfZ. 

Test  Event_ 

Test  Date 


13  Jul 


Recorder  Name: 


Comments 


i<ovr 

v6  vv  ^5*" 

Tlr* /tf.sctn*  4  Iciiij  SC<*  M/CL  Gt>€  tiQ 

/6  jr 

r~-  J  •  ,  yc  ifz  SJ  </£.  i7.'rV  **■  £  -  t  l 

hr^  ?*.  u:fi  . 

ran 

3:?o:f*r  /  /£ p(  .oJV 

ilio 

^  .  j 

c  i/w^  ?:<rr.n  1  p/o  :o>z? 

mi , 

j,  Jo.  :?f ,  HL'-fo-.TT- 

Firr  aACIM  Du/  I 

19  n 

KjC'o^ui  ^loz.SI  /  1 9 :  l  l'.oZz=~ 

ib/L 

r*  ?o.?7./r t</irw7  **"7 

(~rf  lifts'  ' 

.W  o 

JdMtf  ftjoio*  /  ( g:  <90  lO^’Zr- 

(4lO 

.  jc  iy.iT  *c>:ri.rn  A  <  fe 

hrfuC-A- 

\W 

(Uj  £:o9-oo j 

111° 

fp  ^  ?o:  3  <:n,  s'y  t  i)^ 

ZObf 

SMv  PAp^ 

zo  I  o 

iranrararaH 

?0  ll 

3  0*7  i'Q  Z~  pJLs 

■z°T8 
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Appendix  D:  Section  Dl:  AFATDS  Message  Traffic 


JUL-0B-S8  16:20  FROM : D+SA  BATTLE  LAB  I D • 4054427 140 

07  08  98  13  16  CFF.mof 


FACE 


37, 1/14, AMC, , , AC0012, N30. 50213, E046. 25779, 30:30:08,1),  046: 15:28, 

44, A/1/14, AMC, ,,AC0012,N30. 50213, E046. 25779,  30: 30:08, N, 04  6: 15: 2 

52, l/l/A/1/14, AMC, , , AC0012,N30. 50213, E046. 25779,  30: 30: 08, N, 046: 

09, 1/14, WR, , ,AC0012,N30- 50213,  E04 6. 25779, 30 : 30: 08, N, 046:15:28, E 
17,A/1/14,WR, ,,AC0012,N30. 50213, E046. 25779, 30: 30:08, N, 046: 15: 28 

24, l/l/A/1/14, WR,,, AC0012,N30. 50213,  E046. 25779,  30 :30: 08, N, 04  6:1 

12, 1/14, AMC, , , AC0015,N30. 85090, E046. 2058 4, 30 :51:03,N, 046:12:21, 


D,  19:24:18, , ,NA, 18: 43:NA,AC0012,N30 . 50102,  E04  6. 25725,  30 : 30 : 04, N, 046: 15: 
26, E 

C,  19: 24 : 19, A/1/14, AMC,,, AC0015, N30. 85090, £046. 20584, 30: 51: 03,  N, 046: 12:2 
1,E 

C,  19:24:27,  l/l/A/1/14, AMC, , , AC0015,N30. 85090, E046. 20584, 30: 51: 03, N, 046: 
12:21, E 

C,  19:  36: 21, 1/14, FIRE, , ,AC0015, N30. 8S090,E046. 20584, 30: 51: 03, N, 04  6: 12:21 
,E 

C,  19 : 36:29, A/1/14,  FIRE, ,,AC0015,N30. 85090,  E046. 20584, 30: 51: 03, N, 046: 12: 
21, E 

C,  19 : 36. •  36, l/l/A/1/14, FIRE,, ,AC0015,N30. 85090, E04  6. 20584, 30: 51:03, N, 046 

: 12:21, E 

0,19:41:11,,, NA, 1$ : 43 : NA, AC0015, N30 .85245,E046.20427,30:51:09,N,046:12: 
15, E 


I 

h 


t 

n 
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Appendix  D:  Section  D2:  Example  of  Non  Entity  State  PDUs  Generated  by  Fort  Sill 


D  2 


PDU  Type 

PDUJime 

(msec) 

Log_Time 

(msec) 

Difference 

(msec) 

* 

Transmitter 

48175347 

48175515 

168 

Signal 

:  48175347 

48175558 

211 

Transmitter 

!  48175999 

48177141 

1142 

Signal 

:  48175999 

48177297 

1298 

Transmitter 

48175999 

48177796 

1797 

Transmitter 

48177947 

48178110 

163 

Transmitter 

58387247 

58388880 

1633 

Transmitter 

58389647 

58391280 

1633 

firenO  1 8-043-0 1 8  locn:(  30.180 

Fire 

58392247 

58393883 

1636 

47.182)  mg:  164310.0 

Transmitter 

!  58392547 

58394180 

1633 

Transmitter 

:  58395747 

58397381 

1634 

Transmitter 

i  58395747 

58397381 

1634 

Transmitter 

58397447 

58399081 

1634 

Transmitter 

58398047 

58399681 

1634 

Transmitter 

58401247 

58402881 

1634 

Transmitter 

:  58401247 

58402882 

1635 

Transmitter 

58402947 

58404582 

1635 

Transmitter 

58407947 ; 

58409582 

1635 

Transmitter 

58410247 

58411883 

1636 

Transmitter 

58410247 

58411883 

1636 

Transmitter 

58411947 

58413583 

1636 

Transmitter 

58415347; 

58416990 

1643 

Signal 

58415347; 

58417019 

1672 

Transmitter 

58416999 

58418578 

1579 

Signal 

58416999 

58418734 

1735 

Transmitter 

58417999 

58419233 

1234 

Transmitter 

58421047 

58422684 

1637 

Detonation 

58722301 

58723982 

1681 

firer:  0 1 8-043 -0 1 8  result:  Detonation 

Transmitter 

58997999 

58999579 

1580 

Signal 

58998999 

59000541 

1542 

Transmitter 

58998822 

59000571 

1749 

Signal 

58998822 

59000573 

1751 

Transmitter 

59000447 

59002168 

1721 

Transmitter 

59003847 

59005568 

1721 

Transmitter 

59006147 

59007869 

1722 

1  Transmitter 

i  59006147 

59007869 

1722 

Transmitter 

:  59007847 

59009569 

1722 

Transmitter 

59011247 

59012969 

1722 

■  Transmitter 

:  59013747 

59015470 

1723 

Transmitter 

59013747 

59015470 

1723 

Transmitter 

59016147 

59017870 

1723 

Transmitter 

59067947 

59069677 

1730 

Transmitter 

59070247 

59071978 

1731 

Transmitter 

59070247 

59071978 

1731 

Transmitter 

59071947 

59073678 

1731 

Transmitter 

59075347 

59077084 

1737 

Signal 

:  59075347 

59077109 

1762 

Transmitter 

59076999 

59078697 

1698 

Signal 

i  59077999 

59078853 

854 

Transmitter 

;  59077999 

59079352 

1353 

I  Transmitter 

59080347 

59082079 

1732 

|  Transmitter 

59090947 

59092681 

1734 

Transmitter 

59094147 

59095881 

1734 

Transmitter 

59094147 

59095882 

1735' 

Transmitter 

59095847 

59097581 

1734 

D3 
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Appendix  D:  Section  D3:  Fort  Sill  Site  Logs 


JUL-  14-90  13:21  FROM.D  +  SA  BATTLE  LAB 


ID: 4054427140 


Date: 

Location: 

Test  Event: 

Test  Participants: 


ETE  Site  Checklist  (Fort  Sill) 

To-L  ?  ljlir±  -  'T^ 

_ *nr  J/*-*-  _ ' 


to . 


Data  File  System/Name:  g^Ur/V-r--f-  Mc>j  s. 


_ ( . Kiyi&r^fc.  JL 


Test  Start  Time: 
Scenario  Start  Time: 


.  Test  Stop  Time; 


/t:/< p 


J  fry  Scenario  Stop  Time:  ^  :37  1  J  ? 

T.WI  I  •Ao-,‘>'a- 


^■C^^Everi) t". 

|  Network  Activation  (site  activities  in  support  of  JADS  N&E)  | 

1 

System  Reset 

<?* 

2 

Connectivity  Check 

)  Vjoo 

G* 

3 

Log  File  Checks 

Co 

4 

Phone  Checks 

IZ'.OQ 

C  0 

5 

Time  Synchronization 

G>  « 

WWW. 

|  Bring  up  system  (for  each  component  at  site)  | 

1 

See  AFATDS  check  list 
page  3 

(S'  <S> 

2 

See  TAFSM  check  list 
page  4 

^  o 

3 

See  P1U  check  list  page  4 

y  2. 

)  3  :*o 

✓ 

S.4~"l_  r,U 

1  *3  :ro 

fl.'ro 

S-U.-+- 

(*U<4 

7o 

Hut 

*  3:5^/ 

t  3N* 

>V:/? 

'l  VS  i 
?3:  i5C 

S-hf/ 

57 

ll'.t  7 

S^Of 

- 

-*£>  ;  o  5k 

o? 

ole  2 
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7/S 


JUL- 14 


-90  13*22  FROM  *  D  +  SA  BATTLE  LAB 


ID*4054427140 


PAGE  0/9 


Date: 

Location: 

Test  Event: 

Test  Participants: 


ETE  Site  Checklist  (Fort  Sill) 


Jult*  (O  (<?9?  ~~  T-y>;- 

— ft  T'ezr-  _ 

f.TJLAzJ. _ _ _ 


e*  _ _ 


Ff-t>^ig6 


Data  File  System/Name:  A  *■*»/+*&*, 

- - - - 1 »  lO  *>  t  Dir-  1 1  !*« 

- - - jf-jaul _ g . . 

Test  Start  Time:  /  3  ;  0  1 _ Test  Stop  Time:  / 

Scenario  Start  Time:  /  ^  /  g _ _  Scenario  St6p  Time:  .2,0  f  3.  f 


.^.POG; 

SksK^Event  MSMi  ^‘ki“Go«««o  J 

|  Network  Activation  (site  activities  in  support  of  J ADS  N&E)  j 

1 

System  Reset 

/3r  V7 

£. 

2 

Connectivity  Check 

3 

Log  File  Checks 

‘IS 

4 

Phone  Checks 

1 3:00 

5 

Time  Synchronization 

/  J  i  ©  5 

<?« _ 

wmm$ 

i'  PQG  •  J'?5-:i'~”;:Aiiion  : 

??:■?.  "Event 

Bring  up  s 

ystem  (for  each  component  at  site)  | 

1 

See  AFATDS  check  list 
page  3 

/  2  f  S~& 

<a0 

2 

See  TAFSM  check  list 
page  4 

Co 

3 

See  PIU  check  fist  page  4 

uturr 

(s  (p 

5  rtu?  * 

TA-Pi**  i 

SW»-  f  13;  i? 


*.v«  c*~pU*J  -  7  ***<  ?) 


S4C/a  'TACJkv, 

S-ity,  piu/rc^dk.  ao:  a. ; 

Shp\oj^s  -Zo'.ai  ^46  y  /^-vj 
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JUL  — 1 4  —  90  13  =  22  FROM : D  +  SA  BATTLE  LAB 


ID;4054427140 


PAGE 


3/9 


i 


Date: 

Location: 

Test  Event: 

Test  Participants: 


ETE  Site  Checklist  (Fort  Sill) 

A  A*  f  -  X  t ?  6  ft-*^^c£.2P£t..£x 

..Ft*  - StJSd^at <*M _ 

jfr,  _ __ 

£Qi:~t< _ ___ _ _ 

<r+*s _ _ _ 


Data  File  System/Name: 


Test  Start  Time: 
Scenario  Start  Time: 


1 3 >Q  l _  Test  Stop  Time:  /  3 'o^ 

13U0 _  Scenario  Stop  Time:  2 .01(0 


i 


S£3&SSy*nt:-;$:ffiSs& 

[  Network  Activation  (site  activities  in  support  of  JADS  N&E)  J 

I 

System  Reset 

tifY-S 

C~o 

2 

Connectivity  Check 

WOt 

C0 

3 

Log  File  Checks 

73 :  •i  7 

<£„ 

4 

Phone  Checks 

fj-.vd 

C„ 

5 

Time  Synchronization 

/  3'Ot 

C  O 

r„:h*£  rr.LU-,  w.«.  C-V-f^o  - 
<x^eL  %  q  X-Ler-  £ j c cA  a^g1* &  oa^dis. « 


Bring  up  s 

ystem  (for  each  component  at  site)  j 

1 

See  AFATDS  check  list 
page  3 

13:  OO 

Q  o 

2 

See  TAFSM  check  list 
page  4 

/  vl  i? 

&  o 

3 

See  PIU  check  list  page  4  j 

to:  n 

(o  o 

o  o~A~ 


i 


3 hyjer-  13-  /  O 

5-U^  TAFSw  l^:Xo 


£ 


*2  +•  Jrturyds  <£>-v\ 

Sy^  I 

h*'SJ's  *’•* 


AT±C{*S,  rrjr~H:*3  ** 


TAPS* 

S.o.o^ 

C<i[UA  ‘,a  *((  ^r^Jrt  Jo 
i  r  ■  r 

;u>_'  to 

Sr  *+  t*-*  ^ 

7-0  -  fJL 

A  *  * 

Sto:  SST 

D7 


Appendix  E:  Example  Comparison  File  of  PDUs  Broadcast  by  Janus  and  Received  by  VSTARS 


FDU  Comparison  (Grumman-WSMR  Loggers)  (Hour  4  PDUs  16001-32000) 


Game 

Game 

Mean  Median  Max  Min 

Log  Time 

Log  Time 

Apparent 

Time 

Time 

Entity  ■ 

Entity  Apparent  Apparent  Apparent  Apparent 

PDUs  Percent 

(iTlSSC) 

(msec) 

(msec) 

(WSMR)  (Grumman)  Latency  Latency  Latency  Latency 

Lost  Loss 

(WSMR) 

(Grumman) 

(msec) 

(WSMR) 

(Grumman) 

(msec)  (msec)  .  (msec)  (msec) 

59548612 

59548680 

68 

12142902 

12142902' 

3905 

3905  81.4156  80  104!  68 

8  0.0500% 

59548612 

59548684 

72 

12142902 

12142902 

3906 

3906 

59540612 

59548688 

76 

12142902 

12142902 

3907 

3907  | 

59548612 

59548688 

76 

12143234 

12143234 

3909 

3909 

59548612 

59548692 

80 

12143234 

12143234 

3910 

3910 

59548612 

59548696 

84 

12143234 

12143234 

3911 

3911 

59548612 

59548696 

84 

12143234 

12143234 

3912 

3912 

59548612 

59548696 

84 

12143568 

12143568 

3914 

3914 

59548612 

5954B700 

88 

12143568 

12143568 

3915 

3915 

59548612 

59548704 

92 

12143568 

12143568 

3916 

3916 

59548612 

59548704 

92 

12143568 

12143568 

3917 

3917 

59549620 

59549688 

68 

12143902 

12143902 

3919 

3919 

59549620 

59549688 

68 

12143902 

12143902 

3920 

3920 

59549620 

59549692 

72 

12143902 

12143902 

3921 

3921 

59549620 

59549696 

76 

12143902 

12143902 

3922 

3922 

59549620 

59549696 

76 

12144234 

12144234 

3924 

3924  |  ..  Js  ,v 

59549620 

59549700 

80 

12144234 

12144234 

3925 

3925 

59549620 

59549704 

84 

12144234 

12144234 

3926 

3926 

59549620 

59549704 

84 

12144234 

12144234 

3927 

3927 

59549620 

59549704 

84 

12144568 

12144568 

3929 

3929 

59549620 

59549708 

88 

12144568 

12144568 

3930 

3930 

59549620 

59549712 

92 

12144568 

12144568 

3931 

3931 

59549620 

59549712 

92 

12144568 

12144568 

3932 

3932 

59550736 

59550804 

68 

12144900 

12144900 

3934 

3934 

59550736 

59550808 

72 

12144900 

12144900 

3935 

3935 

• 

59550736 

59550808 

72 

12144900 

12144900 

3936 

3936 

59550736 

59550812 

76 

12144900 

12144900 

3937 

3937 

59550744 

59550816 

72 

12145234 

12145234 

3939 

3939 

59550744 

59550816 

72 

12145234 

12145234 

3940 

3940 

59550744 

59550820 

76 

12145234 

121 45234 : 

3941 

3941 

59550744 

59550824 

80 

12145234 

12145234 

3942 

3942 

1  | 

59550744 

59550824 

80 

12145568 

12145568 

3944 

3944 

59550744 

59550824 

80 

12145568 

12145568 

3945 

3945 

59550744 

59550828 

84 

12145568 

12145568 

3946 

3946 

59550744 

59550832 

88 

12145568 

12145568 

3947 

3947 

i  j 

59551612 

59551680 

68 

12145900 

12145900 

3949 

3949 

59551616 

59551684 

68 

12145900 

12145900 

3950 

3950  >  ] 

59551616 

59551688 

72 

12145900 

12145900 

3951 

3951 

:  | 

59551616 

59551688 

72 

12145900 

12145900 

3952 

3952  I 

1  j 

59551616 

59551692 

76 

12146234 

12146234 

3954 

3954 

59551616 

59551696 

80 

12146234 

12146234 

3955 

3955 

i  i 

59551616 

59551696 

80 

12146234 

12146234 

3956 

3956 

59551616 

59551700 

84 

12146234 

12146234 

3957 

3957 

59551616 

59551700 

84 

12146568 

12146568 

3959 

3959 

59551616 

59551704 

88 

12146568 

12146568 

3960 

3960 

59551616 

59551704 

88 

12146568 

12146568 

3961 

3961 

59551616 

59551708 

92 

12146568 

12146568 

3962 

3962 

59552616 

59552684 

68 

12146900 

12146900 

3964 

3964 

59552616 

59552688 

72 

12146900 

12146900 

3965 

3965 

59552616 

59552688 

72 

12146900 

12146900 

3966 

3966 

59552616 

59552692 

76 

12146900 

12146900 

3967 

3967^ 

59552616 

59552696 

80 

12147234 

12147234 

3969 

3969 

. ; 

59552616 

59552696 

80 

12147234 

12147234 

3970 

3970 

59552616 

59552696 

80 

12147234 

12147234 

3971 

3971'  1  J-'A 

59552616 

59552700 

84 

1  12147234 

12147234 

3972 

3972 

59552616 

59552704 

88 

12147566 

12147566 

3974 

3974  .  „  : 

59552616 

59552704 

88 

12147566 

12147566 

3975 

3975  .  j 

59552616 

59552708 

92 

'  12147566 

12147566 

3976 

3976  | 

59552616 

59552712 

96 

12147566 

12147566 

3977 

3977  ■ 

59553612 

59553680 

68 

:  12147900 

12147900 

3979 

3979  ‘  . 

59553612 

59553684 

72 

'  12147900 

12147900 

3980 

3980' 

59553612 

59553688 

76 

12147900 

12147900 

3981 

3981 

. . . .  • 

El 
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Appendix  F:  Operators  Observers  Logs 


ETE  Melbourne  July  7-13, 1998 

This  is  sort  of  a  summary.  I  still  have  my  raw  notes  for  more  detail. 

General  comments; 

The  TSS’s  are  by  MOS,  MI  people,  but  they  really  do  not  do  MI  on  this  Joint 
STARS  job.  That  is,  they  maintain  a  list  of  Radar  Service  Requests  and  they  keep 
the  SMO  straight,  but  they  really  do  not  routinely  take  that  one  step  further  and  even 
attempt  to  do  real  intel.  That  was  sort  of  good  news  for  us  because  they  really 
enjoyed  attempting  to  do  that  job  for  us  in  its  entirety.  They  could  coordinate  more 
with  the  GSM  troops  and  do  more  of  an  intel  job  routinely,  but  they  hesitate  to  talk 
to  GSM.  I  suppose  that  their  job  is  much  more  challenging  with  multiple  GSMs,  but 
on  this  one  GSM  scenario(as  opposed  to  multiple  GSMs),  they  do  not  have  much  to 
do. 

Again,  GSM  and  TSS  share  duties  somewhat,  but  the  GSM  troops  do  most  of  the 
real  intel.  However,  the  TSSes  are  pretty  good,  perhaps  because  they  mostly 
operate  without  GSMs  in  training.  They  mostly  personally  know  each  other  (the 
GSM  troops)  from  some  intel  school  that  they  went  to  together.  There  are  two  types 
of  messages  which  they  use  to  communicate-  free  text  and  UHF  voice.  They  hesitate 
to  employ  UHF  in  real  life.  Therefore  they  hesitate  to  use  it  here  for  us(the  bat 
phone). 

Monday  13  July(last  day) 

Game  started  at  1015L 

All  times  on  this  sheet  local.  Game  went  on  to  almost  1615L 

1020  2A3 1  some  sort  of  movement  NE  to  SW.  Seems  to  split  at  SW  comer  and 
follow  two  columns  like  they  might  be  reading  for  attack  or  dispersing  to  avoid 
getting  killed. 

R&M  1020  Monday  13  Jul.  Tobey  says  that  SARs  are  not  full  size  and  not  getting  a 
label  with  them.  Fixed  later,  I  guess. 

R&M  1030  Monday:  GSM  had  to  go  down  but  brought  it  back  up  in  about  ten 
minutes. 


FI 


R&M  1030  Monday:  time  line  does  not  function?? 

1036  Noted  vehicles  parked  bottom  half  of  the  area 

1036  On  right  side  of  area  31,  another  unit  moving  North  to  south  off  road.  Convoy 
moves  SW  then  scoots  over  to  road. 

1110  That  weird  barrier  blocking  road  with  one  mover  apparently  stopped  by  it. 
Barrier  is  huge,  like  a  building 

1 1 15  no  change  in  anything  lately 

R&M  1120  Monday  13  Jul.  SARs  down  for  about  5  to  10  minutes. 

General:  hour  two,  no  big  change  in  area  3 1  activity  but  area  used  a  lot  less 
General:  Area  34,  general  movement  NW  to  SE  along  road  (10  vehicles  move 
through  a  specific  area  in  A34  from  1 108  to  1115) 

General  1 130-1 145  convoy  moving  south  to  North  on  same  route 
General:  Some  helicopters  flew  north  out  of  an  area  north  of  2A32 

1305  Noticed  big  star  at  UTM  38RPV5613485825(not  in  assigned  area) 

General  hour  3:  same  type  of  activity  along  highway 

Hour  4  1315-1415  Local.  All  attention  given  to  2A35  Generally  everybody  moving 
from  center  outbound.  Jonathan  is  requesting  and  getting  SARs  (remember  that  he  is 
a  simulated  GSM  today)  he  thinks  that  he  found  a  HQ  at  38RNV88417845W0 

Found  armor  battalion  moving  at  bottom  of  GRCA. 

1405  Found  division  or  Brigade  at  38RPU8165571952(not  in  2A35,  but  noticed  it 
anyway).  No  activity  in  204  and  215(subsets  of  A33) 

1430  Big  buildup  at  38RNU9707271506 

1515  Big  Battle.  Major  NW  movements  rest  of  GRCA  but  small  movements 
elsewhere.  Main  supply  of  Hamerabi  moving  NE  to  SW 
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1527  Units  crossed  bridge  at  about  1525.  Also  some  crossing  where  there  are  no 
bridges.  SAR  definitely  does  not  find  a  bridge.  A35  big  time  movement  to  north 
1545  local  stopping  hear  and  there(maybe  due  to  lost  PDUs. 


Friday  10  JUL 

General  background,  big  war  day.  Vignette  number  3.  Activity  stars  at  1019L.  But 
we  say  1015. 

Dilbert  is  the  SMO.  He  is  not  a  real  SMO  but  instead  part  of  the  JTF. 

Big  picture  stuff 

NA  14  has  huge  non  realistic  movement  at  about  1650Z(1250  L) 

General  movement  to  south  all  day  is  evidence  of  a  build  up 

14  had  convoys  leave  during  1st  hour 

no  activity  in  11-12  last  hour(1300-1400L,  I  think) 

no  activity  11,12  last  hour 

no  activity  13  3rd  hour 

looks  like  big  activity  at  Jabila(However,  not  marked  as  active  airbase) 

starts  at  about  1800Z  Go  into  city  at  about  1854  out  of  city  at  about  1858Z 
(comment;  Jonathan  and  Tobey  had  to  go  to  a  meeting  about  1210.  Only  Chris  was 
left  for  rest  of  the  days) 


Blow  by  Blow 

1 045L  bunch  of  activity  starts  out  from  a  little  spot 
NAI  2A-3 1  convoy  moves  NE  to  SW  stopping  in  the  area 
1 100L  SMO  seems  intentionally  slow  (comment  the  TSS’s) 

1 105L  some  activity  in  bottom  edge  of  GRCA 
1 1 15L  some  river  crossing  activity,  but  seems  too  early 
1 1 16L  big  maneuvers  south  of  GRCA 

1423L  2 A3 1  4(?)  convoys  entered  from  NE.  Passed  through,  did  not  stop 
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SARed  the  area  and  found  3  different  targets. 

One  defense  perimeter  of  about  10  vehicles  about  500  meters  across. 

Area  34;  SARed  the  whole  area.  Some  very  strange  thing,  straight  line  (East 
West)blocking  major  route  of  travel  Maybe  this  is  some  kind  of  compound. 

1230L  Some  comm  problem  about  did  you  shoot  SARs  34  and  35  yet  My  TSSs 
confess  to  a  lack  of  understanding  about  the  numbering  system.  Found  another 
highway  west  of  the  original  highway  with  lots  of  activity. 

2A  32  no  activity  across  Euphrates  river  bridge. 

2A-3 1  possible  SAM  site.  Never  did  find  definite  air  defense  site 

Troops  had  to  leave  for  meeting.  That  means  Tobey  and  Jonathan  went  to  Joint 

STARS  meeting.  Chris  was  here  to  do  the  whole  job. 

141 0L  found  4  assembly  areas  south  side  of  GRCA 

1430L  still  movement  on  bottom 
R&M  1430L  SAR  stuck  on  screen 
143 1L  claims  can  observe  actual  attack 

1500L  Chris  asks  if  these  are  real  SARs  of  real  Iraq  taken  from  real  plane. 

1455L  finds  alert  aircraft  on  alert  pad(Not  a  pad  really,  but  a  bunch  of  high  speed 
taxiways) 

1500L  general:  seems  like  everything  up  North  is  dull.  However,  major  battle  at 
Southern  border. 

1 500+  2 A3 1 ,  small  convoy  came  around  southern  half  of  it 

1530L  confirms  with  GSM  that  nothing  happening  first  half  hour  except  skirmishes 

at  S  border 

approx.  1 540  broken  scope 
1 545  scope  back 

1550  crossed  the  bridge  as  radar  came  back  up 

General:  everything  is  going  north 

1615  end  of  game(Gary  calls  “radar  hard  broke”) 


Debrief 
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AFATDS  data  base  corrupted 

First  two  days  mish  mash  (blow  by  blow)  (NO  map  underlays  all 
day  first  day) 

All  times  local 

1131  found  about  a  dozen  targets 

1133 

1134  Waiting  for  TSS  to  reset  SCDL 

1139  All  four  targets  in  area  12  just  stopped 

1145  GSM  reports  that  all  messages  are  getting  rejected(SAR  requests  and  free 
text) 

Big  picture(general):  Between  1100  and  1200  a  HQ  element  or  a  couple  of  log 
convoys  went  out  of  NAI  12  in  two  directions. 

1213  Talil  airfield;  Helo  activity  originated  here  but  no  evidence  of  an  actual  airfield 
(remember,  no  map  underlays  first  day) 

1 250  Bunch  of  MTIs  left  NAI  1 4 


General:  looks  like  the  FLOT  is  about  the  32nd  parallel 
????  GSM  went  “down” 

1257  GSM  finally  came  up.  RSR  50  (the  number  of  a  radar  service  request) 
approved.  Will  transmit  about  50  old  SARs  to  the  GSM. 

1302  voice  comm,  established  with  GSM 


Simisms(“simism”  is  something  on  the  simulation,  due  to  the  “D”  or  the 
“S,”  I  guess,  that  is  different  than  real  life. ) 


FS 


General  agreement  that  SARs  are  much  to  bright  and  star-like.  Reported  directly 
from  Jonathan  to  Lt.  Col.  McCall. 

1036  Monday  13  Jul.  On  right  side  of  area  31,  another  unit  moving  North  to  south 
off  road.  Convoy  moves  SW  then  scoots  over  to  Road.  This  and  others  might  be 
because  of  lost  PDUs 

1036  Monday  13  Jul  Complaint  that  rivers  shown  to  be  much  too  wide.  However, 
maybe  because  of  the  lost  PDU  problem,  being  shown  as  crossed  without  a  bridge. 
If  they  are  that  deep  ought  to  at  least  slow  down  at  the  ridge  or  in  the  soft  bottom(if 
they  are  dry) 


Fence  around  ammo  dump  appear  much  too  thick  and  high(with  big  shadow) 

Generally  way  too  much  clutter  in  SARs.  Too  many  little  cultural  dots,  I  think. 

Jonathan  says  that  the  terrain  looks  like  tree  tops.  It  is  supposed  to  be  low  rocky 
area.  Would  be  a  better  comparison  if  we  made  SAR  images  of  terrain  that  he  is 
used  to. 

Asked  Jonathan  if  he  would  like  to  use  this  machine  to  train  people.  Jonathan  said 
that  it  is  good  for  MTI  but  not  good  for  SAR  training. 

Seems  to  be  too  much  time  between  time  SAR  is  supposed  to  be  shot  and  the  time 
that  it  is  shown  on  screen.  Tobey  remarked  about  this  first  day,  but  he  seems  to  have 
recanted.  The  time  was  about  10-30  seconds  vice  about  10  seconds,  but  might  have 
been  his  polling  scheme. 

Too  much  gap  shows  between  adjacent  SARS  (noted  1115  Monday  13  Jul) 

RADAR  sights  ought  to  look  peculiar  because  of  bright  reflector.  However,  no  sign 
of  this.  That  is  a  SAM  sight  should  occasionally  show  its  radar  antenna 
perpendicular  to  line  between  Joint  STARS  and  target.  We  no  not  see  this. 


R&M  means  some  sort  of  breakdown  or  problem  that  gets  fixed(not 
intrinsic  to  V-STARS) 

R&M  About  noon  Friday,  Tobey  lost  history  function.  Claims  that  does  not  happen 
in  real  aircraft.  Somehow,  mysteriously  fixed  later. 

R&M  1430L  Friday  SAR  stuck  on  screen 

R&M  approx.  Friday  1540  broken  scope 
1 545  scope  back 


R&M  About  1300  first  day,  system  is  deleting  RSRs.  We  do  not  know  why. 

R&M  1300L  Friday  dropped  two  SARs.  Comment  that  is  much  better  than  previous 
days.  Did  not  keep  good  track  of  this  on  previous  days. 

R&M  Monday  13  Jul.  Toby  complained  about  all  computer  functions  being  very 
slow.  Maybe  8  seconds  for  computer  to  react  to  anything.  Mysterious  fix  later  on. 

R&M  1020  Monday  13  Jul.  Tobey  says  that  SARS  not  full  size  and  not  getting  a 
label  with  them.  Fixed  later  I  guess 

R&M  1030  Monday  GSM  had  to  go  down  but  brought  it  back  up  in  about  ten 
minutes. 

R&M  1030  Monday  GSM  had  to  go  down  but  brought  it  back  up  in  about  ten 
minutes. 

R&M  1030  Monday  time  line  does  not  function?? 

R&M  1 120  Monday  13  Jul.  SARs  down  for  about  5  to  10  minutes. 
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R&M  1030L  on  Monday  13Jul  FTI  is  broken 

None  of  the  ellipse  stuff(?)  works)(noted  Monday  13  Jul 

Delays  in  displays  all  gone  (back  to  normal)  Monday  13  Jul ,  noon-ish 


Appendix  G: 

Section  Gl: 

ETE  Test 

Functionality  and  Integration  #4 
Analysis  Summary 

During  the  ETE  Functionality  and  Integration  (F&I)  Test  #4,  analysis  was  performed  to  address 
JADS  measures  dealing  with  the  network,  nodes,  protocol  data  units  (PDUs),  and  test  control. 
The  measures  addressed  and  the  results  obtained  are  provided  below. 

Network 

Measure  2-1-2-2:  Number  of  trials  canceled  or  otherwise  not  used  due  to  network 
problems. 

Measure  2-1-3-3:  Number  of  trials  in  which  network  connection  was  lost  long  enough  to 
require  trial  cancellation 

Measure  2-1-3-6:  Amount  of  down  time  due  to  network  failures. 

Intent:  These  measures  was  used  to  identify  the  impacts  of  network  problems  on  the  F/I. 

Description:  During  the  F&I,  entity  state  PDUs  (ESPDUs)  were  broadcast  from  White  Sands 
Missile  Range  (WSMR)  to  Melbourne  via  the  test  Control  and  Analysis  Center  (TCAC),  and 
entity  state,  fire,  detonate,  transmit,  and  signal  PDUs  were  broadcast  from  Fort  Sill  to  WSMR  via 
the  TCAC. 

Results: 


Date 

Hours 

Scheduled  for 
Testing 

Hours 
Network 
Unavailable 
for  Testing 

%  of  Time 
Network 
Unavailable 

Reason  Unavailable 

25  Jun  98 

5  hrs,  23  mins 

37  mins 

11.46% 

37  mins  -  Power  spikes  at 

WSMR  caused  IDNX  restart 

26  Jun  98 

7  hrs,  32  mins 

36  mins 

7.96  % 

35  mins  -  Ethernet  interface/ 
router  could  not  see  the 
network  at  Fort 

Hood/Grumman 

1  min  -  channel  service 
unit/data  service  unit 
(CSU/DSU)  at  Fort 
Hood/Grumman  went  down 

Gl- 


Total 


12  hrs,  55  mins 

1  hr,  13  mins 

9.42  % 

The  failure  caused  by  the  power  spike  was  a  result  of  an  excessive  number  of  components  being 
plugged  in  to  the  uninterruptible  power  supply  (UPS)  at  WSMR.  This  problem  has  been  resolved 
by  removing  components  from  the  UPS.  The  failures  occurring  on  26  Jun  resolved  themselves. 
Network  and  Engineering  is  investigating  the  causes  of  these  failures. 

Measure  2-1-2-3:  Average  and  peak  bandwidth  used  by  link. 

Intent:  This  measure  was  used  to  provide  an  indication  of  bandwidth  use  during  F&I  4. 

Description:  During  the  operational  test  (OT),  Spectrum™  was  used  to  measure  bandwidth 
utilization  on  the  classified  links  connecting  TCAC  to  Grumman  and  Grumman  to  Fort  Hood. 

The  Spectrum™  network  analysis  tool  provides  the  capability  to  study  multiple  aspects  of 
network  link  performance,  including  packet  rate,  load  (%bandwidth  utilized),  packet  error  rate, 
and  packet  discard  rate.  During  testing.  Spectrum™  logs  these  data  at  a  frequency  specified  by 
the  user.  For  F&I  4,  this  specified  frequency  was  once  every  5  minutes  during  both  days  of 
testing. 

The  following  performance  parameter  definitions  are  used  in  the  reporting  and  discussion  of  the 
bandwidth  data : 

-  Packet  Rate  -  Ratio  of  change  in  total  packet  traffic  (in  and  out)  to  change  in  time 

Load  -  Ratio  of  change  in  total  packet  traffic  to  change  in  time  expressed  as  a 
percentage  of  internal  bandwidth 

-  Error  Rate  -  Ratio  of  bad  packets  to  total  packets  for  elapsed  time  interval 

-  Discard  Rate  -  Ratio  of  discarded  packets  to  total  packets  for  elapsed  time  interval 

The  following  pieces  of  equipment  were  monitored  to  determine  the  bandwidth  data: 

-  JADS  router  (Serial_IF_Port5)  T-l  Line  from  TCAC  to  Grumman 

-  GRUMMAN  router  (Serial_IF_Port3)  T1  Line  from  TCAC 

-  GRUMMAN  router  (Serial_IF_Port4)  T-l  Line  to  Fort  Hood 

Results:  The  following  tables  show  average  and  maximum  values,  respectively,  for  each  of  the 
Spectrum™  performance  parameters  for  the  classified  network  links  monitored  during  active  ETE 
F&I  testing  and  for  each  of  the  performance  parameters  for  the  classified  network  links  monitored 
during  and  immediately  following  active  F&I  testing. 
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NETWORK  AVERAGE  PERFORMANCE  CHARACTERISTICS* 


DATE/ 

TIME 

NODE 

A 

NODE 

B 

PACKET 

RATE 

LOAD 

ERROR 

RATE 

DISCARD 

RATE 

25  Jun  98 
1440:1857 

TCAC 

GRM 

16.306/ sec 

.94% 

.016% 

0% 

25  Jun  98 
1440:1857 

GRM 

FT 

HOOD 

21.47/sec 

.067% 

0% 

26  Jun  98 
1334:2037 

TCAC 

GRM 

18.03/ sec 

0% 

0% 

26  Jun  98 
1334:2037 

GRM 

FT 

HOOD 

20.81/sec 

.28% 

.117% 

0% 

*  Table  refers  only  to  active  test  time  during  which  PDU  loggers  were  recording  data. 


NETWORK  MAXIMUM  PERFORMANCE  CHARACTERISTICS* 


DATE/ 

TIME 

NODE 

A 

NODE 

B 

PACKET 

RATE 

LOAD 

ERROR 

RATE 

DISCARD 

RATE 

25  Jun  98 
1440:1857 

TCAC 

GRM 

27/ sec 

mm 

0% 

25  Jun  98 
1440:1857 

GRM 

FT 

HOOD 

62/ sec 

4% 

■si9 

0% 

26  Jun  98 
1334:2037 

TCAC 

GRM 

22/ sec 

2% 

0% 

0% 

26  Jun  98 
1334:2037 

GRM 

FT 

HOOD 

64/ sec 

4% 

8% 

0% 

*  Table  refers  only  to  active  test  time  during  which  PDU  loggers  were  recording  data. 


NETWORK  AVERAGE  PERFORMANCE  CHARACTERISTICS** 


DATE/ 

TIME 

NODE 

A 

NODE 

B 

PACKET 

RATE 

LOAD 

ERROR 

RATE 

DISCARD 

RATE 

25  Jun  98 
1330:2200 

TCAC 

GRM 

13.24/sec 

1.22% 

.028% 

0% 

25  Jun  98 
1300:2200 

GRM 

FT 

HOOD 

13.78/ sec 

.128% 

1.18% 

0% 

26  Jun  98 
1330:2200 

TCAC 

GRM 

15.9/ sec 

1.23% 

0% 

0% 

26  Jun  98 
1330:2200 

GRM 

FT 

HOOD 

18.48/sec 

231% 

.183% 

0% 

**  Table  refers  to  all  time  during  which  SPECTRUM  was  logging  data,  during  and  immediately 


following  the  test. 
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NETWORK  MAXIMUM  PERFORMANCE  CHARACTERISTICS** 


DATE/ 

TIME 

NODE 

A 

NODE 

B 

PACKET 

RATE 

LOAD 

ERROR 

RATE 

DISCARD 

RATE 

25  Jun  98 
1330:2200 

TCAC 

GRM 

180/ sec 

68% 

2% 

0% 

25  Jun  98 
1300:2200 

GRM 

FT 

HOOD 

62/ sec 

4% 

63% 

0% 

26  Jun  98 
1330:2200 

TCAC 

GRM 

32/ sec 

0% 

0% 

26  Jun  98 
1330:2200 

GRM 

FT 

HOOD 

64/ sec 

9% 

0% 

**  Table  refers  to  all  time  during  which  SPECTRUM  was  logging  data,  during  and  immediately 
following  the  test. 

Based  on  the  data  used  in  the  compilation  of  the  above  tables,  the  following  conclusions  were 
reached  on  the  load/packet,  error,  and  discard  rates  for  F&I  4: 

Load/Packet  Rate.  Packet  rate  experienced  for  both  classified  link  portions  averaged  around  20 
packets/second.  Network  load  (bandwidth  utilized)  during  active  test  execution  averaged  around 
1%,  and  peak  load  never  exceeded  5%  of  link  capacity  during  the  two-day  testing  period. 
However,  packet  rate  and  network  load  did  increase  significantly  over  the  TCAC  -  Grumman  link 
immediately  following  active  testing  each  day.  This  increase  can  be  explained  by  the  use  of  the 
network  links  immediately  following  active  testing  to  import  PDU  logger  data  files  to  the  TCAC 
from  the  distributed  Grumman  site.  Peak  loads  for  this  link  during  the  time  immediately  following 
test  execution  ranged  from  42%  to  68%  of  network  capacity  over  the  two  days.  Recorded  packet 
rate  decreases  on  the  second  day  of  testing  correspond  to  known  network  outages  and  VSTARS 
problems. 

Error  Rate.  Error  packets  were  experienced  infrequently  during  the  two  days  of  testing.  The 
packet  error  rate  average  hovered  around  1%  for  both  classified  link  portions  during  active 
testing.  There  were  several  occasions  during  which  a  large  number  of  error  packets  were 
recorded  within  the  span  of  a  few  logging  intervals.  A  63%  maximum  error  rate  for  one  interval 
was  the  largest  recorded,  occurring  over  the  Grumman  -  Fort  Hood  link  after  active  testing  was 
completed. 

Discard  Rate.  The  average  packet  discard  rate  experienced  during  each  day  of  testing  was  zero. 
No  packets  were  discarded  during  any  day  of  testing. 

Nodes 


Measure  1-2-3-1:  Degree  to  which  advanced  distributed  simulation  (ADS)  can  add  assets 
to  test  execution. 

Intent:  This  measure  was  intended  to  document  how  many  assets  ADS  could  add  to  test 
execution  during  the  F&I. 
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Description:  For  the  F&I,  Janus  scenarios  were  used  to  simulate  entities  and  movements  for  a 
corps  area  of  operations. 

Results: 


Date 

Scenario 

Entities 

25  Jun  98 

Vignette  1  (901) 

9766 

26  Jun  98 

Vignette  2  (902) 

9766 

See  unclassified  attachment  for  entity  by  type. 

Measure  2-1-3-1:  Number  of  trials  delayed,  rescheduled,  and/or  redone  due  to  ADS 
components  exclusive  of  network  unavailability. 

Measure  2-1-3-5:  Number  of  ADS  system  failures  (severe  enough  to  require  trial 
cancellation). 

Intent:  These  measures  addressed  the  availability  of  ADS  systems,  including  network  interface 
units  (NIUs)  during  F&I  testing. 

Description:  For  the  F&I,  the  ADS  system  consisted  of  Janus  at  WSMR,  Virtual  Surveillance 
Target  Attack  Radar  System  (VSTARS),  Tactical  Army  Fire  Support  Model  (TAFSM)  at  Fort 
Sill,  and  the  common  ground  station  (CGS)  at  Fort  Hood. 
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Results: 


Date 

Location 

Hours 
Scheduled 
for  Testing 

Hours 
Unavailable 
for  Testing 

%  of  Time 
Node 

Unavailable 

Reason  Unavailable 

25  Jun  98 

WSMR 

5  hrs, 

23  mins 

37  mins 

11.46% 

Power  spikes  at  WSMR 
caused  IDNX  restart 

Hood 

5  hrs, 

23  mins 

0 

0% 

Sill 

6  hrs 

0 

0% 

Grumman 

5  hrs, 

23  mins 

0 

0% 

26  Jun  98 

WSMR 

7  hrs, 

32  mins 

0 

0% 

Hood 

7  hrs, 

32  mins 

36  mins 

7.96  % 

Ethernet  interface/  router 
could  not  see  the  network 
and  the  CSU/DSU  went 
down. 

Sill 

7  hrs, 

25  mins 

0 

0% 

Grumman 

7  hrs, 

32  mins 

36  mins 

7.96  % 

Ethernet  interface/  router 
could  not  see  the  network 
and  the  CSU/DSU  went 
down. 

Totals 

WSMR 

12  hrs, 

55  mins 

37  mins 

4.77  % 

Hood 

12  hrs, 

55  mins 

36  mins 

4.65  % 

Sill 

12  hrs, 

55  mins 

0 

0% 

Grumman 

12  hrs, 

55  mins 

36  mins 

4.65  % 
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Testing  for  less  than  1  hour  of  25  Jun  was  scrubbed  because  of  a  power  spike  which  incapacitated 
ETE  testing  ability  at  WSMR.  Both  Fort  Hood  and  Grumman  experienced  problems  on  26  Jun, 
with  either  a  router  or  the  Ethernet  interface  causing  the  problem.  Fort  Sill  did  not  experience 
any  problems  on  either  day. 

PDU  Analysis 

Measure  2-1-2-5:  Percentage  of  PDUs  received  out  of  order  by  a  network  node. 

Measure  2-1-2-6:  Percentage  of  total  PDUs  required  at  a  node  that  were  delivered  to  that 
node. 

Measure  2-1-2-7:  Average  and  peak  data  latency  between  ADS  nodes. 

Intent:  Measure  out  of  order,  lost  PDUs,  duplicate  PDUs,  PDU  latency,  variability,  and  rates. 

Description:  During  the  F&I,  entity  state  (ESPDUs)  PDUs  were  broadcast  from  WSMR  to 
Melbourne  via  the  TCAC,  and  entity  state,  fire,  detonate,  transmit,  and  signal  PDUs  will  be 
broadcast  from  Fort  Sill  to  WSMR  via  the  TCAC. 


Results: 


Date 

Node 

A 

Node 

B 

PDUs  sent 

PDUs  rec’d 

PDU 
rate  per 
second 

PDUs  lost 
total /  % 

Duplicate 

PDUs 

PDUs  out 
of  order 

Min/  max/  mean 
latency 

6/25 

w 

T 

188,805 

164,671 

12.36 

24,134 

12.78% 

0 

0 

.018/.1 18/.030 

T 

G 

164,671 

164,657 

10.72 

14 

.009% 

0 

0 

.030/.045/.033 

S 

W 

233 

187 

.015 

46 

19.74% 

0 

0 

-.35/-. 03/-. 03 
(see  Note  1) 

6/26 

W 

T 

308,916 

308,898 

12.35 

18 

.006% 

0 

0 

.017/.139/.038 

T 

G 

308,898 

293,258 

11.72 

15,640 

5.06% 

0 

0 

.028/.060/.035 

S 

W 

437 

437 

.017 

0 

0% 

0 

0 

-.39/-.03/-.03 
(see  Note  1) 

Total 

w 

T 

497,721 

473,569 

12.36 

24,152 

4.85% 

0 

0 

.017/.  1 39/.034 

T 

G 

473,569 

457,915 

11.22 

15,654 

3.31% 

0 

0 

.028/.060/.034 

S 

W 

670 

624 

.016 

46 

6.87% 

0 

0 

-.39/-. 03/-. 03 
(see  Note  1) 

G  -  Grumman  S  -  Fort  Sill  T  -  TCAC  W  -  WSMR 

Note  1  -  Logger  clocks  were  not  synchronized,  resulting  in  negative  latencies. 
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Analysis  of  Network  Impact  on  PDUs 


Date 

Node 

A 

Node 

B 

PDUs 

sent 

Network-related  PDU  losses 

Unexplained  PDU  losses 

%  of  PDUs  sent  /  %  of  PDUs 
lost 

%  of  PDUs  sent  /  %  of  PDUs 
lost 

PDUs 

lost 

6/25 

W 

T 

188,805 

24,134 

0 

24,134 

100%  /  100% 

0%  /  0% 

T 

G 

164,671 

0 

14 

14 

0%  /  0% 

.009%  / 100% 

S 

W 

233 

0 

46 

46 

0%  /  0% 

19.74%  / 100% 

w 

T 

308,916 

0 

18 

18 

0%  /  0% 

.006%  / 100% 

T 

G 

15,611 

29 

S 

W 

437 

0 

N/A 

N/A 

Total 

w 

T 

497,721 

24,134 

18 

24,152 

4.85%  /  99.93% 

.004 

%  /  .07% 

T 

G 

473,569 

15,611 

43 

15,654 

s 

46 

6.87%  / 100% 

PDU  losses  on  both  test  days  were  significant.  Most  of  the  losses  were  due  to  router  and  trunk 
outages  resulting  in  a  loss  of  connectivity  of  the  end-to-end  loop. 


Measure  2-2-1-1:  Degree  to  which  ADS  nodes  provide  for  collection,  data  entry,  and 
quality  checking  of  pre-  and  post-trial  briefing  data. 
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Measure  2-2-1-4:  Ease  with  which  data  can  be  retrieved,  post  trial,  from  a  given  node. 

Intent:  This  measure  was  used  to  provide  a  mechanism  for  capturing  unforeseen  problems  with 
data  collection,  data  entry,  and  quality  checking. 

Description:  During  the  F&I,  automated  data  were  collected  using  PDU  loggers  at  the  nodes 
and  operational  data  were  collected  using  log  sheets  at  each  node. 

Results: 

The  data  collected  from  the  F&I  test  were  retrieved  in  a  fairly  simple  fashion.  All  data  were  sent 
via  file  transfer  protocol  (FTP)  from  the  test  locations  involved  to  JADS.  These  data  were  then 
compressed  and  converted  for  analysis  on  JADS'  UNIX-based  analysis  tools.  While  the  data 
retrieval  was  simple,  it  was  not  entirely  effective.  Data  from  the  Grumman  site  were  not 
completely  converted  by  the  conversion  software,  with  this  incomplete  conversion  transparent  to 
the  analysts  upon  initial  review.  However,  based  on  a  similar  experience  from  the  previous  ETE 
Test,  the  problem  was  discovered  after  the  data  analysis  was  initiated.  The  conversion  process 
was  then  repeated  for  the  Grumman  data,  with  the  second  conversion  completed  successfully. 

This  should  not  be  a  problem  during  future  test  events,  as  past  experience  will  alert  the  analysts  to 
such  data  conversion  problems. 

Test  Control 


Measure  2-2-2-1  Degree  to  which  test  managers  can  control  the  configurations  of  ADS 
participants,  the  ADS  environment  data,  and  ADS  networks 

Intent:  This  measure  is  intended  to  determine  if  the  test  manager  can  adequately  control  the  test 
configuration  of  ADS  participants,  the  ADS  environment  data,  and  ADS  network  both  during 
and  between  test  events. 

Description:  During  the  test  readiness  review,  the  baseline  configuration  was  verified.  During 
the  test,  the  test  controller  documented  all  procedures  used  to  control  the  test  configuration, 
environment  data,  and  ADS  network,  along  with  what  procedures  were  useful,  and  which  ones 
needed  modification. 


Measure  2-2-2-1  Degree  to  which  test  managers  can  control  the  configurations  of  ADS 
participants,  the  ADS  environment  data,  and  ADS  networks 

Intent:  This  measure  is  intended  to  determine  if  the  test  manager  can  adequately  control  the  test 
configuration  of  ADS  participants,  the  ADS  environment  data,  and  ADS  network  both  during  and 
between  test  events. 

Results: 

Procedures  used  to  execute  the  test  included  Test  Controller  Checklist  and  Network  Coordinator 
Checklist.  Network  procedures  provided  positive  control  of  the  network  and  data  collection  by 
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personnel  using  computer  loggers  at  remote  sites.  Test  control  procedures  allowed  monitoring  of 
critical  equipment  and  provided  necessary  communication  capabilities. 
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Appendix  G: 

Section  G2:  Operational  Measures  Report 


For  the  seven  below  listed  tasks  and  MOPS,  a  JADS  observer  sat  next  to  the  OWS  operator  at 
Northrop  Grumman  and  the  GSM  operator  at  Fort  Hood  and  asked  them  to  perform  the  tasks. 
The  operator  was  instructed  to  look  anywhere  in  the  GRCA  and  report  when  he  saw  any  below 
listed  activities.  Performing  these  tasks  were  secondary  to  the  operator’s  normal  function  of 
conducting  surveillance  and  providing  intelligence  to  the  TAC.  The  operator  was  not  informed 
that  a  certain  type  of  activity  was  occurring  at  a  certain  location  at  a  certain  time.  Instead,  he  was 
merely  told  to  locate  certain  type  of  activities  and  provide  information  about  it.  The  tasks, 
operator  information,  and  the  analysis  performed  are  listed  below. 

Task  1:  Identify  an  Airfield 

Operator  Information:  Vignette  1 


Game  Time 

Location 

Activity 

6:46 

PV04552455 

Random  nonfrequent 
movement  observed  over 
several  hours  j 

6:49 

PU53838057 

Random  nonfrequent 
movement  observed  over 
several  hours 

Analysis:  The  operators  were  able  to  identify  airfields  based  on  their  electronic  map  (EMAP) 
and  not  the  moving  target  indicator  (MTI)  feed  received  from  VSTARS.  Their  observations  on 
the  activity  around  the  airfield  were  based  on  the  MTI  feed,  and  they  were  able  to  use  synthetic 
aperture  radar  (SAR)  to  confirm  the  existence  of  an  airfield  and  enemy  activity.  The  presence  of 
an  airfield  and  enemy  activity  was  confirmed  by  looking  at  the  movement  tables  for  enemy 
activity.  During  the  OT,  this  task  will  be  changed  from  identify  airfields  to  identify  activity 
on/around  airfields. 

Task  2:  Identify  Artillery 

Operator  Information:  Operators  were  not  able  to  identify  artillery  units  during  either  Vignette 
1  or  Vignette  3.  According  to  the  operators,  artillery  units  do  not  look  any  different  from  other 
units  when  moving  in  a  convoy  down  an  msin  supply  route  (MSR).  When  cued  by  other  sources, 
they  have  an  improved  probability  of  confirming  artillery  locations  as  the  batteries  move  into  a 
firing  positions.  It  is  very  difficult  to  identify  artillery  units  unless  the  operator  continually 
observes  one  or  several  units  throughout  the  entire  scenario. 

Analysis:  During  the  OT,  JADS  analysts  will  cue  the  operators  to  observe  certain  units  during 
certain  time  frames.  The  operators  will  not  be  informed  that  they  are  looking  for  artillery  units. 
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JADS  analysts  will  try  to  determine  if  the  operators  can  identify  artillery  units  based  on  size, 
speed,  and  movement.  Information  provided  by  the  operator  will  be  documented. 

Task  3:  Identify  Assembly  Areas 


Operator  Information:  Vignette  1 


Game  Time 

Location 

Activity 

6:53 

PV16651329 

Approximately  25  units 
located  in  the  area. 

Analysis:  Operator  observation  was  verified  using  movement  tables.  According  to  the 
movement  tables,  a  brigade  headquarters  with  8  combat  battalions  was  located  at/around 
PV16651329.  This  included  a  total  of  22  companies. 

Operator  Information:  Vignette  3 


Game  Time 

Location 

Activity 

5:55 

PU29239828 

Approximately  25  units 

5:57 

PU42679633 

Approximately  26  units 

Analysis:  Movement  tables  confirmed  that  at  5:55,  an  armor  regiment,  2  motorized  rifle 
regiments,  and  various  support  units  were  located  at  PU29239828.  Movement  tables  also 
confirmed  that  at  5:57,  a  brigade  headquarters  and  a  maneuver  battalion  were  located  at 
PU42679633. 

Task  4:  Identify  Convoys 


Operator  Information:  Vignette  1 


Game  Time 

Location 

Activity 

6:44 

PV72492721 

10  wheeled  vehicles  moving 
east  at  38  kph 

Analysis:  Movement  tables  indicated  that  the  nearest  convoy  at  6:44  at/around  PV72492721 
was  at  PV705283  and  contained  22  wheeled  vehicles.  The  operator  information  was  inaccurate 
for  convoy  size  and  location.  A  review  of  the  entire  vignette  revealed  that  no  convoys  moved 
at/around  PV72492721  for  the  duration  of  the  entire  vignette.  The  nearest  convoy  was 
approximately  2  kilometers  (km)  away. 
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Operator  Information:  Vignette  3 


Game  Time 

Location 

Activity 

5:58 

PU45399304 

Approximately  19  wheeled 
vehicles  moving  SE  at 

25kph 

6:01 

PU59599133 

Approximately  16  wheeled 
vehicles  moving  SW  at 

20kph 

Analysis:  Movement  tables  revealed  that  neither  of  the  convoy  locations  were  correct.  The 
nearest  convoy  from  the  first  observation  was  approximately  3  kms  away  from  the  actual  location. 
The  nearest  convoy  from  the  second  observation  was  approximately  8  kms  away  from  the  actual 
location. 

Task  5:  Identify  Stationary  Targets 
Operator  Information:  Vignette  3 


Game  Time 

Location 

Activity 

6:15 

PU45399304 

Approximately  1 9  wheeled 
vehicles  moving  SE  at 

25kph 

6:19 

PU59599133 

Approximately  16  wheeled 
vehicles  moving  SW  at 

20kph 

Analysis:  Movement  tables  revealed  that  neither  of  the  convoy  locations  were  correct.  The 
nearest  convoy  from  the  first  observation  was  approximately  3  kms  away  from  the  actual  location. 
The  nearest  convoy  from  the  second  observation  was  approximately  8  kms  away  from  the  actual 
location. 

Task  6:  Identify  Logistics  Sites 
Operator  Information:  Vignette  1 


Game  Time 

Location 

Activity 

6:37 

PV1028 

Approximately  7  convoys 
moving  out  of  logistics  site 

6:37 

PV5582 

Major  log  site; 
approximately  10  convoys 
moving  in/out  of  log  site 

Gl-3 


Analysis:  Movement  tables  confirmed  that  at  6:37  game  time,  PV1028  and  PU5582  were 
operating  as  log  sites.  PV1028  contained  the  corps  logistics  headquarters  with  four  company  size 
elements.  PU5528  was  a  major  assembly  area  and  logistics  site  containing  25  units  to  include  the 
division  headquarters,  air  defense  artillery  (ADA)  units,  artillery  units,  and  logistics  units. 


Operator  Information:  Vignette  3 


Game  Time 

Location 

Activity 

5:56 

PV073160 

Convoys  moving  in  and  out 
of  log  site 

5:56 

PU427787 

Convoys  moving  in  and  out 
of  log  site 

Analysis:  Movement  tables  revealed  that  PV073160  was  a  log  site  containing  a  truck  company 
with  225  vehicles.  However,  PU427787  was  not  a  log  site.  The  closest  log  site  was 
approximately  7  kms  away.  A  separate  ADA  battalion  (Bn)  was  located  approximately  2  kms 
away. 

Task  7:  Identify  Main  Supply  Routes  (MSR) 


Operator  Information:  Vignette  1 


Game  Time 

Location 

Activity 

6:35 

PV1369 — PV2524 

6  convoys  moving  N  to  S 
over  1  hour 

6:36 

PV6353 — PV2637 

5  convoys  moving  NW  to 

SE  over  1  hour 

Analysis:  Movement  tables  confirmed  that  there  were  numerous  convoys  traveling  north  to 
south  on  an  MSR  from  PV1369  to  PV2524  .  However,  there  was  no  MSR  or  convoys  moving 
from  PV6353  to  PV2637.  The  nearest  MSR  at  this  point  was  at  PV4547  to  PV7627.. 


Operator  Information:  Vignette  3 


Game  Time 

Location 

Activity 

6:03 

PU1088 — PU2019 

Convoy  route 

6:10 

PU2998 — PU4095 

Convoy  route 

Analysis:  Movement  tables  revealed  that  neither  of  these  sites  were  MSRs.  Though  there  was 
scattered  movement  from  PU1088  to  PU2019,  the  movement  was  sporadic  and  limited.  There 
was  not  enough  movement  to  label  it  an  MSR.  PU2998  to  PU4095  were  5  kms  away  from  the 
nearest  MSR. 
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Measure  of  Performance  (MOP)  9:  Maintain  link  between  CGS  and  VSTARS 

During  the  test  readiness  exercises,  there  were  numerous  failures  in  our  ability  to  conduct  link  ups 
between  VSTARS  (Grumman)  and  the  ground  station  module  (GSM)  (Fort  Hood).  Our  initial 
approach  was  to  set  up  monitoring  tools  to  determine  if  the  problem  was  with  VSTARS,  the 
GSM,  or  the  network  connection.  Due  to  time  constraints  and  cost  considerations,  the  ETE  Test 
team  has  decided  to  look  at  this  link  as  one  entity  instead  of  three  separate  components  during  the 
Phase  2  OT.  As  a  result,  we  will  report  on  this  as  part  of  the  network  analysis  for  JADS  measure 
2- 1-3 -6. 

MOP  11:  Capacity  to  maintain  tracks 

During  the  risk  reduction,  GSM  operators  were  able  to  successfully  maintain  tracks  on  all 
convoys  that  they  attempted  to  maintain  tracks  on.  During  Vignette  1,  operators  initiated  and 
maintained  29  tracks  during  the  six-hour  scenario  without  any  problems.  A  questionnaire  was 
also  used  to  assess  operators’  perception  of  their  ability  to  maintain  tracks.  The  results  are  as 
follows: 

Question:  Rate  the  adequacy  of  VSTARS  in  tracking  moving  ground  targets. 

Respond:  Total  respondents-6.  Extremely  effective  3/6  Very  effective  3/6 

MOP  12:  Ability  to  provide  multimode  radar 

A  questionnaire  was  used  to  determine  if  the  GSM  operators  had  any  problems  with  receiving 
different  types  of  radar  (MTI,  SAR,  fixed  target  indicator  [FTI]).  A  log  was  also  maintained  on 
the  types  and  quantity  of  radar  reports  requested  and  received.  Results  are  as  follows: 


Vignette 

SAR 

Requested 

SAR  Received 

FTI  Requested 

FTI  Received 

1 

86 

232 

2 

2 

3 

81 

535 

1 

1 

JADS  analysts  did  not  observe  any  problems  in  the  operators’  ability  to  request  or  receive  the 
different  types  of  radar  reports.  Results  of  questionnaire  are  as  follows: 

Question:  Rate  the  effectiveness  of  VSTARS  to  interleave  (execute  both  operations  at  the  same 
time)  MTI  and  SAR  modes  of  operation. 

Response:  Total  respondents-5.  Extremely  effective  1/5  Very  effective  4/5 

MOP  15:  Operations  and  control  (O&C)  capabilities  for  VSTARS 
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The  intent  of  this  MOP  was  to  ensure  that  the  operators  could  use  the  workstation  with  VSTARS 
as  they  would  with  Joint  Surveillance  Target  Attack  Radar  System  (STARS)  and  confirm  that 
VSTARS  did  not  cause  any  unforeseen  technical  disruptions  or  anomalies.  A  questionnaire  was 
used  to  determine  if  the  GSM  and  operator  workstation  (OWS)  operators  had  any  difficulties  in 
using  their  workstations  with  VSTARS.  The  results  of  the  questionnaires  are  as  follows: 

Question:  Rate  the  adequacy  of  VSTARS  in  responding  to  your  immediate  requests  for 
information 

Response:  Total  -6.  Extremely  effective  1/6  Very  effective  5/6 

Question:  Rate  the  effectiveness  of  the  surveillance  control  data  link  (SCDL)  (T-l  line)  in 
transferring  free  text  message  between  the  VSTARS  and  GSM 

Response:  Total  -5.  Extremely  effective  2/5  Very  effective  3/5 

Question:  Rate  the  effectiveness  of  the  SCDL  in  transferring  radar  service  request  (RSR) 
messages  between  the  GSM  and  VSTARS 

Response:  Total-5.  Extremely  effective  1/5  Very  effective  2/5  Somewhat  effective  1/5 
Borderline  1/5 

Question:  Rate  the  adequacy  of  VSTARS  in  tracking  moving  ground  targets 

Response:  Total-6.  Extremely  effective  3/6  Very  effective  3/6 

Question:  Rate  the  adequacy  of  E-8  O&C/GSM  console  in  displaying  MTI  radar  data 

Response:  Total-5.  Extremely  effective  1/5  Very  effective  2/5  Somewhat  effective  2/5 

Question:  Rate  the  adequacy  of  E-8  O&C/GSM  console  to  process  and  display  SAR  radar 

Response:  Total-5.  Extremely  effective  1/5  Very  effective  3/5  Borderline  1/5 

Question:  Rate  the  adequacy  of  E-8  O&C/GSM  console  in  displaying  FTI  radar  data 

Response:  Total-5.  Very  effective  3/5  Somewhat  effective  1/5  Borderline  1/5 

Question:  Rate  the  effectiveness  of  the  E-8  O&C/GSM  console  in  displaying  map-like  features 
(terrain,  roads,  and  other  cartographic  data)  while  in  the  MTI  mode  of  operation 

Response:  Total-4.  Extremely  effective  1/4  Very  effective  1/4  Somewhat  effective  1/4 
Somewhat  ineffective  1/4 
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Question:  Rate  the  effectiveness  of  the  E-8  O&C/GSM  console  in  displaying  map-like  features 
(terrain,  roads,  and  other  cartographic  data)  while  in  the  FTI  mode  of  operation 

Response:  Total-3.  Extremely  effective  1/3  Very  effective  2/3 

MOP  16  SAR  integration  function  time 

Post  risk  reduction,  we  determined  that  the  SAR  integration  time  is  a  developmental  test  (DT) 
measure  and  not  OT.  The  SAR  integration  time  for  OT  is  a  function  of  how  long  the  airborne 
target  surveillance  supervisor  (ATSS)  sits  on  a  SAR  request  before  sending  it  to  the  system 
management  officer  (SMO)  and  how  long  the  SMO  sits  on  the  request  before  granting  it.  Any 
data  gathered  for  this  as  OT  measure  will  be  irrelevant  since  the  wait  time  for  a  SAR  could  be 
anywhere  from  1  minute  to  30  minutes  or  even  longer.  As  a  DT  measure,  the  integration  time 
would  begin  from  the  time  the  SMO  approved  a  SAR  until  the  time  it  was  produced.  Since  this  is 
a  set  parameter  based  on  the  platform  that  VSTARS  is  running  on,  it  was  measured  during  the 
DT  and  will  not  change  for  the  OT. 
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